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1.0  INTRODUCTION 

This  report  documents  the  results  of  an  Air  Force  Research  Laboratory  Information 
Directorate  (AFRL/IF)  in-house  program  titled  “Routing  For  Next  Generation  MILSATCOM”. 
The  report  is  divided  into  several  distinct  sections  corresponding  to  the  effort’s  main  thmsts. 
While  this  program  was  in  existence  for  several  years,  its  objectives  changed  over  time  and  the 
end  result  is  several  distinct  efforts  performed  under  the  umbrella  of  this  one  effort.  Since  this 
effort  was  in  existence  for  several  years,  the  vast  majority  of  the  work  documented  here  was 
actually  performed  by  AFRL/IF’ s  organizational  predecessor,  Rome  Laboratory.  However, 
Rome  Laboratory  technically  no  longer  exists  and  any  references  to  Rome  Laboratory  that  were 
in  previous  drafts  of  this  report  have  been  replaced  with  AFRL/IF  as  a  result. 

When  this  effort  began,  its  objective  was  to  develop  strategies  for  routing  messages  in 
the  next  generation  mihtary  sateUite  communications  (MILSATCOM)  environment.  The 
developed  routing  strategies/algorithms  were  to  be  prototyped  and  tested  in  computer 
simulations  to  determine  their  performance.  While  this  program  was  redirected  midstream  to 
pursue  other  objectives,  the  partial  results  of  this  portion  of  the  program  are  documented  in 
Section  2  of  this  report. 

At  management’s  direction,  this  program  was  redefined  to  support  theatre  extension 
objectives  of  the  Global  Grid  Program.  Under  this  new  identity,  the  objective  of  this  program 
was  to  investigate  the  feasibihty  of  Global  Grid’s  theatre  extension  objectives.  Through  analysis 
and  simulation,  this  program  was  to  investigate  the  apphcabihty  of  super  high  frequency  (SHF) 
and  extremely  high  frequency  (EHF)  sateUite  resources  to  playing  the  role  of  gateway  in  a  high 
speed  global  network.  Under  these  general  guidelines,  several  subtasks  were  initiated  to 
explore  various  aspects  of  the  concept  of  space  based  asynchronous  transfer  mode  (ATM) 
switches.  Results  of  this  portion  of  the  program  are  documented  in  Section  3  of  this  report. 

Mid- way  through  the  ATM  related  tasks,  another  major  task  was  undertaken. 
AFRL/IF  received  a  request  from  the  Chief  Scientist  of  U.S.  Space  Command 
(USSPACECOM)  to  perform  a  preliminary  vaUdation  and  accreditation  of  their  NUICCS 
Analyst  Technical  Environment  (NATE)  command,  control,  &  communications  simulation 
model  Since  this  model  was  similar  in  many  ways  to  the  simulator  being  used  for  this  in-house 
program  and  the  same  principal  engineer  was  to  be  used,  the  decision  was  made  to  include  the 
NATE  VaUdation  as  a  new  task  in  this  program.  While  a  report  documenting  the  results  of  the 
NATE  Validation  was  prepared  for  USSPACECOM,  the  results  were  never  published  in  an 
AFRL/IF  Technical  Report.  For  this  reason,  the  results  of  the  NATE  Validation  are 
documented  in  Section  4  of  this  report. 

Upon  completion  of  the  NATE  Validation  task,  work  resumed  on  the  ATM  related 
tasks  started  earUer.  While  by  that  time  most  of  the  earlier  planned  ATM  tasks  were  overcome 
by  events,  I  decided  to  add  one  final  task  in  that  area.  Early  on,  I  had  considered  the  use  of  an 
existing  in-house  simulation  model,  the  Multiple  SateUite  System  (MSS)  End-To-End  Simulation 
(ETESIM),  as  a  tool  for  studying  ATM  sateUite  networks.  OriginaUy,  I  had  determined  that 
there  were  limitations  to  the  appUcabiUty  of  our  ETESIM  to  the  ATM  sateUite  network  problem. 
However,  after  further  thought  I  reaUzed  that  a  relatively  smaU  amount  of  effort  could  result  in 
several  smaU  but  meaningful  ATM  related  changes  and  a  version  of  the  ETESIM  could  be 
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created  which  was  hardwired  for  ATM  analysis.  While  I  was  successful  in  creating  this  new 
ATM  version  of  our  ETESIM  tool  and  it  was  a  good  learning  experience  for  me,  it  was 
nonetheless  a  wasted  task  in  the  end.  The  hardware  platform  that  ETESIM  was  hosted  on, 
which  was  already  largely  obsolete  at  the  beginning  of  the  task,  broke  down  altogether  by  the 
time  this  task  was  completed.  Given  the  obsolescence  of  the  hardware,  it  was  not  worth  the 
investment  to  repair  it  and  the  entire  system  was  scrapped.  Since  it  seemed  pointless  to  do  so,  I 
have  not  included  a  separate  section  in  this  report  to  document  this  task. 
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2.0  ROUTING  ALGORITHM  DEVELOPMENT 


2. 1  ROUTING  ALGORITHM  DEVELOPMENT  OVERVIEW 

The  main  objective  of  this  task  was  to  develop  a  simple  mle-based  message  routing 
algorithm  for  sateUite  communications  networks.  Emphasis  was  intentionally  kept  on  making  the 
algorithm  simple  in  operation.  While  the  algorithm  was  never  intended  to  provide  optimal 
solutions  to  any  routing  problem,  its  simple  operation  was  intended  to  greatly  reduce  the 
processing  necessary  to  perform  the  routing  function  in  a  large  communications  network. 

This  task  consisted  of  five  main  parts.  First,  a  candidate  sateUite  communications 
network  was  defined.  Second,  the  concept  of  operation  of  the  routing  algorithm  was  defined. 
Third,  the  algorithm  was  implemented  in  software.  Fourth,  its  performance  in  the  candidate 
communications  network  was  simulated  in  a  static  environment.  Fifth,  its  performance  would  be 
simulated  in  a  dynamic  environment.  The  fifth  subtask  was  never  done  due  to  this  project  being 
redirected  to  pursue  other  objectives. 

While  the  main  objective  of  this  effort  was  to  develop  a  routing  algorithm,  it  also 
resulted  in  a  useful  by-product.  While  implementing  the  algorithm,  it  became  apparent  that  a 
methodology  for  quickly  developing  routing  algorithms  would  also  be  a  product  of  this  effort. 


2.2  CHOOSING  A  CANDIDA  TE  COMMUNICA  TIONS  NETWORK 

The  term  sateUite  communications  network  is  rather  broad.  Communications 
requirements  can  vary  widely  from  network  to  network.  For  this  reason,  before  anything  else 
was  done  under  this  effort,  a  candidate  communications  network  was  chosen  to  design  to. 

In  general,  the  network  would  consist  of  earth  terminals  and  sateUites.  In  conventional 
sateUite  communications  systems,  a  smaU  number  of  sateUites  in  geosynchronous  orbit  are  used 
as  relay  nodes  for  earth  terminal  to  earth  terminal  communications.  Only  a  smaU  number  of 
complex  sateUites  are  needed  in  this  type  of  system,  because  their  altitude  generaUy  provides 
them  with  a  very  wide  field  of  view.  Another  key  characteristic  of  this  type  of  system  is  that  the 
sateUites  are  stationary  with  relation  to  any  point  on  the  earth.  In  this  type  of  system,  there  are 
relatively  few  communications  networking  problems,  essentiaUy  none  in  the  area  of  routing 
algorithms. 

However,  there  have  been  several  sateUite  communications  systems  proposed,  as  well 
as  a  couple  that  have  actually  been  built,  over  the  past  several  years  that  foUow  a  considerably 
different  design  philosophy.  In  these  systems,  the  sateUite  relays  would  be  placed  in  a  much 
lower  orbit  around  the  earth.  The  lower  altitude  of  the  satelUtes  would  mean  that  the  satelUtes 
would  have  a  much  smaUer  field  of  view  and  more  sateUites  would  be  necessary  to  provide 
complete  coverage  of  the  earth.  However,  each  sateUite  would  be  much  less  complex  than  their 
geosynchronous  counterparts  and  the  larger  numbers  would  reduce  the  importance  of  any  single 
sateUite  to  system  operation.  This  would  result  in  a  communications  system  that  could  be  much 
more  survivable. 
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In  this  type  of  system,  the  term  communications  network  suddenly  becomes  very 
apphcable.  Due  to  the  small  field  of  view  of  the  satellites,  messages  sent  over  long  distances  can 
no  longer  be  accomphshed  by  simply  sending  it  up  to  a  sateUite  and  that  sateUite  sending  it  back 
down  to  its  destination.  Instead,  the  message  may  be  relayed  several  times.  This  could  be  done 
by  sending  the  message  up  to  a  satellite,  the  satellite  sending  it  back  down  to  an  intermediate 
earth  terminal,  that  terminal  sending  it  back  up  to  another  satellite,  etc.,  until  the  message  gets  to 
its  destination.  However,  this  could  also  be  done  by  sending  the  message  up  to  a  satellite,  that 
satellite  relaying  it  to  another  satelhte,  etc.,  mtil  the  message  reaches  a  sateUite  which  has  the 
destination  earth  terminal  within  its  field  of  view.  The  latter  is  the  method  that  is  generaUy 
considered  in  this  type  of  system.  Therefore,  since  these  sateUites  wiU  not  be  stationary  with 
respect  to  the  earth,  the  communications  system  in  question  becomes  a  communications 
network  with  a  dynamic  topology. 

Ultimately,  the  future  miUtary  sateUite  communications  architecture  is  intended  to  be  an 
integrated  environment  consisting  of  both  geosynchronous  and  low  earth  orbit  (as  well  as  any 
orbit  in  between)  sateUite  systems  and  including  both  miUtary  and  commercial  space  assets  in  a 
variety  of  operating  frequencies  and  data  rates.  Therefore,  it  was  our  intention  to  ultimately  also 
simulate  any  algorithms  that  we  developed  in  an  integrated  environment  of  this  sort.  However,  it 
is  the  low  earth  orbit  systems  that  are  the  drivers  for  sateUite  communications  routing  algorithm 
development  and  this  was  the  general  type  of  network  configuration  that  was  chosen  for  a 
baseUne  architecture  in  this  effort.  Fairly  arbitrary  numbers  of  100  satelUtes  in  a  low  earth  orbit 
(750  km)  and  10  earth  terminals  were  chosen  as  network  communications  nodes.  To  simpUfy 
the  static  simulation  program,  the  latitudes  and  longitudes  of  aU  sateUites  and  earth  terminals 
were  chosen  at  random  by  the  computer.  This  is  certainly  an  unrealistic  characteristic  for  a 
deployed  system,  but  with  a  sufficiently  large  and  dense  consteUation,  it’s  effective  as  a  cmde 
approximation.  AdditionaUy,  the  transmitters  on  the  sateUites  had  a  maximum  range  of  2000 
km. 

2.3  ROUTING  ALGORITHM  CONCEPT  OF  OPERA  TION 


2.3.1  BASELINE  ROUTING  ALGORITM 

As  was  mentioned  earlier,  a  conscious  effort  was  made  to  make  the  operation  of  the 
algorithm  as  simple  as  possible.  The  algorithm  was  to  feature  distributed  operation.  Entire 
paths  of  messages  did  not  have  to  be  chosen  at  one  time.  Each  network  node  along  a  message 
path  would  have  the  responsibiUty  of  choosing  only  the  next  node  to  send  the  message  to.  This 
is  significant  for  various  reasons.  Eor  one,  the  problem  that  each  node  must  solve  is  now  a  local 
one,  rather  than  a  global  one.  Therefore,  the  problem  should  be  much  simpler  to  solve  and 
require  much  less  information  about  the  rest  of  the  network.  Second,  a  distributed  approach  is 
much  more  tolerant  to  changes  in  network  conditions.  If  network  conditions  change  somewhere 
during  the  course  of  a  message  transmission,  the  entire  original  message  path  does  not  need  to 
be  recalculated.  The  changed  network  conditions  are  simply  accounted  for  in  the  remainder  of 
the  message  path. 
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There  were  slightly  different  versions  of  the  algorithm  depending  on  whether  a 
communications  node  is  an  earth  terminal  or  a  sateUite.  There  were  a  total  of  five  criteria,  or 
conditions,  used  to  determine  the  next  network  node  to  send  any  message  to.  These  five 
conditions  are  hsted  below. 

1 .  Is  the  message  destination  currendy  in  range? 

2.  Is  sateUite  node  No. _ in  the  general  direction  of  the  destination? 

3.  Has  satelUte  node  No. _ already  been  used  for  this  message  path? 

4.  Is  satelUte  node  No. _ isolated  from  the  network? 

5.  Is  satelUte  node  No. _ congested  (message  queue  full  or  nearly  fuU)? 

The  first  conditions  meant  that  if  the  destination  for  a  message  could  be  reached  directly 
by  the  sateUite  that  was  currently  holding  the  message,  relay  nodes  were  not  necessary  at  all  and 
the  message  should  be  sent  directly  to  the  destination.  According  to  the  second  condition,  it 
would  be  preferable  to  not  send  the  message  in  the  opposite  direction  of  the  message.  While 
the  world  is  round  and  the  message  should  get  there  eventually  anyway,  it  made  sense  to  avoid 
this  if  possible.  The  third  condition  forbade  sending  a  message  back  to  a  sateUite  that  it  had 
already  passed  through.  This  was  necessary  in  the  algorithm  to  avoid  a  message  taking  an 

endless  loop  as  a  path.  The  fourth  condition  was  necessary  to  avoid  message  paths  that  were 

dead  ends.  FinaUy,  the  fifth  condition  said  that  it  was  preferable  to  avoid  satelUte  network 
nodes  that  had  message  queues  that  were  fuU  or  nearly  fuU.  This  was  to  help  aUeviate 
congestion  in  the  network  and  to  avoid  creating  bottlenecks  in  the  network. 

As  was  mentioned  earUer,  there  were  actuaUy  two  sUghtiy  different  versions  of  the 
algorithm  for  the  two  types  of  nodes  in  our  network.  In  the  version  of  the  algorithm  intended  for 
the  earth  terminals,  conditions  1  and  3  were  absent.  Condition  1  was  not  used,  because  two 
earth  terminals  could  not  communicate  directly  with  each  other.  Condition  3  was  not  used, 
because  when  the  algorithm  was  being  performed  by  an  earth  terminal,  the  earth  terminal  was 
the  source  of  the  message.  Therefore,  any  sateUite  that  it  might  consider  sending  the  message  to 
next  could  not  possibly  be  one  which  that  message  had  already  passed  through.  The  earth 
terminal  version  of  the  algorithm  is  shown  below.  The  conditions  were  combined  to  form  mles 
and  for  each  message  sent,  each  mle  was  checked  sequentially  in  the  order  Usted  here  until  aU  of 
the  conditions  associated  with  a  mle  were  satisfied  and  the  mle  was  subsequently  triggered.  This 
mle  determined  where  the  message  was  sent  next.  For  each  message,  the  sateUites  in  the  earth 
terminal’s  neighbor  Ust  were  checked  sequentially  until  a  sateUite  relay  was  found  which  satisfied 
the  most  conditions  possible.  The  first  choice  would  be  a  sateUite  that  met  aU  three  conditions. 
If  this  were  not  possible,  the  second  choice  would  be  a  satelUte  that  would  be  in  the  right 
direction,  and  not  be  isolated.  If  this  were  not  possible,  the  algorithm  would  settle  for  the  first 
sateUite  that  wasn’t  isolated.  The  algorithm  is  shown  below  for  a  system  with  n  sateUites. 

R1  If  SateUite  No.  1  is  in  the  right  direction 

SatelUte  No.  1  is  not  isolated 
SateUite  No.  1  is  not  congested 
Then  send  message  to  SateUite  No.  1 


5 


R2 

If 

Satellite  No.  2  is  in  the  right  direction 

Satellite  No.  2  is  not  isolated 

Satellite  No.  2  is  not  congested 

Then 

send  message  to  Satelhte  No.  2 

Rn 

If 

Satelhte  No.  n  is  in  the  right  direction 
SateUite  No.  n  is  not  isolated 

Satelhte  No.  n  is  not  congested 

Then 

send  message  to  Satehite  No.  n 

R(n+1) 

If 

Satehite  No.  1  is  in  right  direction 
Satelhte  No.  1  is  not  isolated 

Then 

send  message  to  Satehite  No.  1 

R(n+2) 

If 

Satehite  No.  2  is  in  right  direction 
Satehite  No.  2  is  not  isolated 

Then 

send  message  to  Satehite  No.  2 

R(2n) 

If 

Satehite  No.  n  is  in  right  direction 
Satehite  No.  n  is  not  isolated 

Then 

send  message  to  Satehite  No.  n 

R(2n+1) 

If 

Satehite  No.  1  is  not  congested 
Satehite  No.  1  is  not  isolated 

Then 

send  message  to  Satehite  No.  1 

R(2n+1) 

If 

Satehite  No.  2  is  not  congested 
Satelhte  No.  2  is  not  isolated 

Then 

send  message  to  Satehite  No.  2 

R(3n) 

If 

Satelhte  No.  n  is  not  congested 
Satehite  No.  n  is  not  isolated 

Then 

send  message  to  Satehite  No.  n 

R(3n+1) 

If 

Satehite  No.  1  is  not  isolated 

Then 

send  message  to  Satelhte  No.  1 

R(3n+2) 

If 

Satehite  No.  2  is  not  isolated 

Then 

send  message  to  Satehite  No.  2 

R(4n) 

If 

Satehite  No.  n  is  not  isolated 
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Then 


send  message  to  Satellite  No.  n 


Since  all  five  conditions  would  need  to  be  used,  the  version  of  the  algorithm  that  would 
be  used  by  each  sateUite  would  be  very  similar,  but  shghtly  more  complex.  This  version  of  the 
algorithm  is  shown  below. 


R1 

R2 


R3 


If  Destination  node  is  reachable  directly 

Then  send  message  to  destination  node 

If  SateUite  No.  1  is  in  right  direction 

SateUite  No.  1  is  not  isolated 
SatelUte  No.  1  was  not  already  used 
SateUite  No.  1  is  not  congested 
Then  send  message  to  SateUite  No.  1 

If  SateUite  No.  2  is  in  right  direction 

SateUite  No.  2  is  not  isolated 
SatelUte  No.  2  was  not  already  used 
SateUite  No.  2  is  not  congested 
Then  send  message  to  SateUite  No.  2 


R(n+1)  If 

Then 

R(n+2)  If 

Then 

R(n+3)  If 

Then 


SateUite  No.  n  is  in  right  direction 
SateUite  No.  n  is  not  isolated 
SateUite  No.  n  was  not  already  used 
SatelUte  No.  n  is  not  congested 
send  message  to  SateUite  No.  n 
SateUite  No.  1  is  in  right  direction 
SateUite  No.  1  is  not  isolated 
SatelUte  No.  1  was  not  already  used 
send  message  to  SateUite  No.  1 
SateUite  No.  2  is  in  right  direction 
SateUite  No.  2  is  not  isolated 
SatelUte  No.  2  was  not  already  used 
send  message  to  SateUite  No.  2 


R(2n+1)  If 


Then 

R(2n+2)  If 


SateUite  No.  n  is  in  right  direction 
SateUite  No.  n  is  not  isolated 
SateUite  No.  n  was  not  already  used 
send  message  to  SateUite  No.  n 
SateUite  No.  1  is  not  isolated 
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Then 

Satelhte  No.  1  was  not  already  used 
send  message  to  SateUite  No.  1 

R(2n-i-3) 

If 

SateUite  No.  2  is  not  isolated 

Then 

Satelhte  No.  2  was  not  already  used 
send  message  to  SateUite  No.  2 

R(3n-i-I) 

If 

SateUite  No.  n  is  not  isolated 

Then 

SateUite  No.  n  was  not  already  used 
send  message  to  SateUite  No.  n 

2.3.2  EXTENSIONS  OF  ALGORITHM  FOR  INTEGRATED  ENVIRONMENT 

In  order  to  extend  the  use  of  this  baseline  algorithm  for  use  in  the  integrated  sateUite 
communications  environment  planned  for  the  future,  an  additional  decision  process  was  needed. 
Before  a  message  was  routed,  a  type  of  satelhte  resource  was  first  selected  based  on  the  nature 
of  the  message  traffic.  Types  of  sateUite  resources  (e.g.  UHF  LEO,  EHF  GEO,  etc.)  were 
considered  rather  than  actual  systems  to  aUow  the  decision  process  to  include  systems  that  did 
not  exist  at  the  time  but  conceivably  could  in  the  future.  Factors  taken  into  consideration  in  this 
decision  process  included  priority  of  the  message,  data  rate  requirements  of  the  message, 
whether  or  not  the  message  was  a  long  voice  transmission,  and  whether  access  to  polar  regions 
was  required.  If  a  message  was  of  low  priority  and  low  data  rate  was  sufficient,  a  UHF  sateUite 
resource  was  chosen  as  a  preference.  If  high  data  rate  was  necessary,  a  SHF  sateUite  resource 
was  chosen  as  a  preference.  If  a  message  was  of  high  priority,  an  EHF  sateUite  resource  was 
chosen  as  a  preference.  In  this  case,  EHF  sateUite  resources  are  assumed  to  be  highly  reliable, 
but  low  data  rate  systems  such  as  Milstar.  If  a  message  was  not  a  long  voice  transmission,  a 
FEO  satelhte  resource  was  chosen  as  a  preference  due  to  the  lower  propagation  delay 
associated  with  FEO  systems.  If  the  message  was  a  long  voice  transmission,  a  GEO  sateUite 
resource  was  chosen  to  avoid  the  need  for  sateUite  hand- offs  in  the  middle  of  the  transmission. 
The  entire  SateUite  Resource  Selector  (SRS)  algorithm  is  shown  below. 


Rulel 


Rule2 


Satellite  Resource  Selector  (SRS)  Algorithm 

If  destination  is  beyond  range  of  geo  sateUite 

message  is  of  low  priority 
low  data  rate  link  is  sufficient 
message  wiU  not  be  a  long  voice  transmission 

Then  preferred  resource  is  a  network  of  UHF  FEO  sats 

If  destination  is  beyond  range  of  geo  sateUite 

high  data  rate  Unk  is  necessary 
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Then 

message  will  not  be  a  long  voice  transmission 
preferred  resource  is  a  network  of  SHF  LEO  sats 

Rule3 

If 

destination  is  beyond  range  of  geo  sateUite 

Then 

message  is  of  high  priority 

message  will  not  be  a  long  voice  transmission 

preferred  resource  is  a  network  of  EHF  EEO  sats 

Rule4 

If 

source  or  destination  is  in  a  polar  region 

Then 

message  is  of  low  priority 

low  data  rate  link  is  sufficient 

preferred  resource  is  a  network  of  UHE  EEO  sats 

Rules 

If 

source  or  destination  is  in  a  polar  region 

Then 

high  data  rate  is  necessary 

preferred  resource  is  a  network  of  SHE  EEO  sats 

Rule6 

If 

source  or  destination  is  in  a  polar  region 

Then 

message  is  of  high  priority 

preferred  resource  is  a  network  of  EHE  EEO  sats 

Rule? 

If 

message  is  a  long  voice  transmission 

Then 

message  is  of  low  priority 

low  data  rate  link  is  sufficient 

preferred  resource  is  a  UHE  GEO  satelhte 

Rules 

If 

message  is  a  long  voice  transmission 

Then 

high  data  rate  is  necessary 

preferred  satelhte  resource  is  a  SHE  GEO  sateUite 

Rule9 

If 

message  is  a  long  voice  transmission 

Then 

message  is  of  high  priority 

preferred  satelhte  resource  is  a  EHE  GEO  sateUite 

RulelO 

If 

UHE  EEO  sateUite  network  is  preferred  resource 

Then 

there  are  no  UHE  EEO  satelUtes  available 
preferred  resource  is  a  UHE  GEO  satelhte 

Rulel  1 

If 

SHE  EEO  satelhte  network  is  preferred  resource 

Then 

there  are  no  SHE  EEO  satehites  available 
preferred  resource  is  a  SHE  GEO  sateUite 

Rule  12 

If 

EHE  EEO  satelhte  network  is  preferred  resource 
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there  are  no  EHF  LEO  satellites  available 
Then  preferred  resource  is  a  EHF  GEO  satelhte 

The  original  routing  algorithm  for  the  earth  terminals  was  then  modified  slightly  to 
incorporate  the  SRS  decision  process  into  the  routing  mles.  The  updated  version  of  the 
resulting  Earth  Terminal  Uphnk  Selector  (ETUS)  algorithm  is  shown  below. 

EARTH  TERMINAL  UPLINK  SELECTOR  (ETUS)  ALGORITHM 

For  an  Earth  Terminal  which  tracks  n  neighboring  sateUites: 

Rule  1  If  Satelhte  No.  1  is  of  preferred  sateUite  resource  type 

Satehite  No.  1  is  in  the  right  direction 
SateUite  No.  1  is  not  isolated 
SateUite  No.  1  is  not  congested 
Then  send  message  to  SateUite  No.  1 

Rule  2  If  Satellite  No.  2  is  of  preferred  satellite  resource  type 

SateUite  No.  2  is  in  the  right  direction 
SateUite  No.  2  is  not  isolated 
SateUite  No.  2  is  not  congested 
Then  send  message  to  SateUite  No.  2 


Rule  n  If  SateUite  No.  n  is  of  preferred  satellite  resource  type 

Satellite  No.  n  is  in  the  right  direction 
SateUite  No.  n  is  not  isolated 
Satellite  No.  n  is  not  congested 
Then  send  message  to  SateUite  No.  n 

Rule  n+1  If  Satellite  No.  1  is  of  preferred  sateUite  resource  type 

SateUite  No.  1  is  in  the  right  direction 
SateUite  No.  1  is  not  isolated 
Then  send  message  to  SateUite  No.  1 

Rule  n+2  If  Satellite  No.  2  is  of  preferred  sateUite  resource  type 

SateUite  No.  2  is  in  the  right  direction 
SateUite  No.  2  is  not  isolated 
Then  send  message  to  SateUite  No.  2 
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Rule  2nlf  Satellite  No.  n  is  of  preferred  satellite  resource  type 

Satellite  No.  n  is  in  the  right  direction 
SateUite  No.  n  is  not  isolated 
Then  send  message  to  Satelhte  No.  n 


Rule  2n+l  If  Satelhte  No.  1  is  of  preferred  sateUite  resource  type 

SateUite  No.  1  is  not  congested 
SateUite  No.  1  is  not  isolated 
Then  send  message  to  SateUite  No.  1 


Rule  2n+2  If  Satelhte  No.  2  is  of  preferred  sateUite  resource  type 

SateUite  No.  2  is  not  congested 
SateUite  No.  2  is  not  isolated 
Then  send  message  to  SateUite  No.  2 


Rule  3nlf  Satelhte  No.  n  is  of  preferred  sateUite  resource  type 

Satelhte  No.  n  is  not  congested 
Satelhte  No.  n  is  not  isolated 
Then  send  message  to  SateUite  No.  n 


Rule  3n+l  If 

Then 


Satelhte  No.  1  is  of  preferred  sateUite  resource  type 
SateUite  No.  1  is  not  isolated 
send  message  to  SateUite  No.  1 


Rule  3n+2  If 

Then 


Satelhte  No.  2  is  of  preferred  sateUite  resource  type 
SateUite  No.  2  is  not  isolated 
send  message  to  SateUite  No.  2 


Rule  4nlf  Satelhte  No.  n  is  of  preferred  sateUite  resource  type 

SateUite  No.  n  is  not  isolated 
Then  send  message  to  SateUite  No.  n 

Once  a  sateUite  resource  type  had  been  chosen  for  a  message  and  a  message  had 
entered  a  given  sateUite  network,  it  was  assumed  that  it  would  remain  in  that  network  until  the 
message  was  deUvered  to  the  destination.  To  not  make  this  assumption  and  to  aUow  a  message 
to  pass  through  multiple  and  diverse  sateUite  networks  would  have  the  impUcation  that  aU 
sateUites  would  be  interoperable  with  each  other,  regardless  of  frequency,  waveform,  data  rate. 
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etc.,  and  that  seemed  unrealistic.  Therefore,  the  hnk  selection  process  for  the  satellites  remained 
unchanged. 

2.4  IMPLEMENTATION 

The  C  programming  language  was  chosen  for  implementing  the  algorithm  due  to  its 
availability  of  a  C  compiler  to  the  project  engineer. 

As  was  intended,  the  algorithm  was  compact  and  quick  to  execute  once  it  was 
implemented.  Only  61  lines  of  C  code  were  necessary  for  the  baseline  earth  terminal  version 
and  only  1 14  tines  were  necessary  for  the  baseline  satellite  version.  Its  quite  likely  that  the  code 
could  have  been  made  even  smaller  and  quicker  by  a  more  experienced  C  programmer.  The 
extensions  for  the  integrated  satellite  environment  were  not  coded  due  to  redirection  of  the 
effort. 

A  large  reason  for  the  simplicity  of  the  code  is  that  lookup  tables  were  used  wherever 
possible.  There  were  two  lookup  tables  needed  for  each  version  of  the  baseline  algorithm. 
Each  version  had  a  table  of  the  latitudes  and  longitudes  of  each  of  the  earth  terminals.  The  size 
of  this  table  depends  on  how  many  earth  terminals  are  in  the  system.  If  there  are  m  earth 
terminals  in  the  system,  the  size  of  the  table  will  be  m  by  2.  Both  the  earth  terminals  and  the 
satellites  will  also  have  a  neighbor  table.  There  are  a  finite  number  of  satellites  that  either  an 
earth  terminal  or  a  satellite  is  in  range  of  at  any  given  time.  Therefore  it  is  not  necessary  to 
consider  every  satellite  in  the  system  as  a  potential  next  node  in  a  message  path.  One  only 
needs  to  consider  those  satellites  that  are  available  at  the  time.  The  size  of  these  neighbor  tables 
is  somewhat  arbitrary.  Since  the  number  of  neighboring  satellites  that  any  given  node  will  have 
win  vary  with  time,  the  simplest  way  to  implement  this  table  is  to  put  an  upper  limit  on  how  many 
neighbors  each  node  will  keep  track  of.  In  the  software  developed  for  this  project,  each  earth 
terminal  kept  track  of  at  most  10  neighboring  satellites  and  each  satellite  kept  track  of  at  most  8 
neighboring  satellites.  Each  entry  in  these  tables  has  four  fields.  Each  entry  includes  a  satellite 
identification  number,  its  latitude  and  longitude,  and  a  flag  indicating  whether  that  satellite  is 
congested  or  not. 

The  contents  of  these  neighbor  tables  would  obviously  vary  with  time  as  the  network 
topology  changes.  However,  all  information  in  these  tables  could  be  provided  by  the 
neighboring  satellites  themselves  during  routine  operation.  Periodically,  each  satellite  would 
scan  with  its  antenna  the  area  around  it  to  determine  which  satellites  are  its  neighbors  and  where 
they  are.  During  these  times,  those  neighboring  satellites  would  also  report  whether  they  are 
currently  congested  with  traffic  or  not  and  whether  or  not  their  own  neighbor  tables  currently 
have  more  than  one  satellite  listed  in  them  (whether  they  are  isolated  or  not).  Each  network 
node  could  then  update  its  own  neighbor  table.  By  not  including  any  satellites  that  have 
indicated  that  they  were  isolated  in  the  table,  the  need  for  the  routing  algorithm  to  check  this  is 
completely  removed. 
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2.5  SIMULATION 

In  order  to  test  the  routing  algorithm  initially,  a  fairly  simple  but  flexible  static  network 
simulation  shell  was  also  developed.  The  simulator  includes  a  series  of  easy  to  use  menus  that 
are  used  to  define  the  ground  segment,  the  space  segment,  the  message  traffic,  the  condition  of 
the  network,  and  the  routing  algorithm  to  be  used.  Since  the  main  purpose  of  this  simulator  was 
to  provide  a  platform  for  rapid  prototyping  of  routing  algorithms  for  space-based 
communications  networks,  there  were  certainly  many  compromises  made  during  its 
development.  IVbst  significantly,  the  success  of  any  message  dehvery  is  solely  a  matter  of 
routing  decisions  made.  Of  course,  in  an  actual  system  the  success  would  also  depend  upon 
factors  such  as  the  bit  error  rate  on  each  communications  link.  Also,  the  simulator  uses  a  static 
network  topology.  Since  the  satellite  systems  in  question  would  actually  be  dynamic  in  nature, 
the  simulations  performed  correspond  essentially  to  performance  of  the  routing  algorithm  during 
snapshots  in  time.  Once  the  algorithm  was  shown  to  be  stable  when  used  in  a  static 
environment,  the  algorithm  was  intended  to  be  ported  to  AFRL/IF’s  Multiple  SateUite  System 
(MSS)  End-To-End  Simulation  (ETESIM)  for  testing  in  a  dynamic  environment 

Static  simulation  results  are  provided  by  graphical  displays.  Examples  of  these  displays 
are  shown  in  Eigures  1  and  2.  Each  figure  shows  a  grid  that  represents  a  rectangular  view  of  the 
entire  Earth.  Eatitude  and  longitude  are  annotated  along  the  axes  of  the  grid.  On  the  grid,  the 
locations  of  earth  terminals  are  indicated  by  red  dots  and  the  locations  of  satellites  are  indicated 
by  blue  dots.  The  intensity  of  the  color  of  the  satelhtes  indicates  the  queue  size  onboard  those 
satelhtes.  Eight  blue  indicates  a  sateUite  which  will  be  considered  to  be  congested  and  dark  blue 
indicates  a  normal  status.  The  magenta  lines  between  network  nodes  indicate  routing  decisions 
made  during  the  simulation.  The  entire  path  for  one  message  is  displayed  at  one  time.  Above 
the  grid,  the  number  of  total  messages,  the  number  of  completed  messages,  and  the  number  of 
lost  messages  are  indicated. 

Eigures  1  and  2  illustrate  the  performance  of  the  routing  algorithm  in  the  same  network 
with  the  same  message.  Eigure  1  shows  the  routing  decisions  made  for  the  case  where  the 
network  is  fuUy  operational  and  there  are  no  congested  nodes.  Eigure  2  shows  the  routing 
decisions  made  for  the  case  where  approximately  30%  of  the  satelhtes  are  currendy  congested. 
In  this  figure,  the  network  congestion  control  feature  of  the  routing  algorithm  is  clearly  shown. 
One  of  the  satelhtes  in  the  message  path  shown  in  Eigure  1  is  now  considered  to  be  congested. 
As  a  result,  a  new  path  is  chosen  to  avoid  the  congested  area.  These  figures  also  emphasize  the 
fact  that  optimum  paths  are  not  being  sought.  The  first  path  which  meets  ah  of  our  criteria  (or  as 
many  as  possible)  is  chosen. 

However,  as  advertised,  the  processing  time  appears  to  almost  negligible.  While  a 
quantitative  figure  for  processing  time  is  currently  unavailable,  the  entire  path  was  calculated  and 
displayed  certainly  faster  than  the  eye  could  fohow  using  a  standard  desktop  personal  computer. 
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RUN  SIMULATION 
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Figure  1  -  Non-Congested  Network  Simulation  Case 
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Figure  2  -  30%  Congested  Network  Simulation  Case 


2.6  METHODOLOGY  FOR  DESIGNING  ROUTING  ALGORITHMS 

While  the  prime  result  of  this  project  is  a  new  routing  algorithm,  it  became  apparent 
during  the  course  of  this  work  that  a  significant  by-product  of  this  project  was  a  methodology 
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for  fairly  quickly  and  painlessly  developing  routing  algorithms  in  general.  A  mle  based 
description  of  the  algorithm's  operation  proved  to  be  a  rapid  and  easily  understood  method  of 
developing  the  top-level  description  of  the  algorithm.  From  this  point,  t  should  be  a  fairly 
straightforward  task  to  implement  the  algorithm  in  an  object-oriented  programming  language.  In 
this  case,  the  C  language  was  used,  but  it’s  likely  that  other  object-oriented  languages  would 
also  be  well  suited  for  such  a  task. 


2.7  CONCLUSIONS 

The  results  of  this  project  demonstrate  the  suitability  of  mle-based  system  concepts  to 
the  problem  of  message  routing  in  a  communications  network.  A  simple  mle-based  routing 
algorithm  was  developed  which  possessed  the  desired  qualities  of  distributed  operation  and  low 
processing  requirements.  The  algorithm  provided  admittedly,  and  intentionally,  sub-optimal 
solutions  with  great  efficiency.  By  the  experience  gained  in  doing  this  project,  it  is  obvious  to 
the  project  engineer  that  this  same  methodology  could  be  easily  be  used  to  provide  a  myriad  of 
solutions  to  the  same  routing  problem.  A  much  more  optimal  solution  could  have  easily 
achieved  at  the  expense  of  additional  processing  and/or  data  storage.  However,  I  believe  that  a 
fuUy  distributed  routing  algorithm  which  also  handles  congestion  control  in  the  network  with 
minimal  processing  and  data  storage  is  still  quite  an  achievement. 
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3.0  ATM  ANALYSIS 


3.1  ATM  ANALYSIS  OVERVIEW 

The  general  objective  of  these  tasks  was  to  provide  modehng  &  simulation/analysis  in 
support  of  theatre  extension  objectives  of  the  Global  Grid  Program.  Global  Grid  was  a 
program  for  developing  technologies  leading  to  a  global  high  speed  communications  network. 
Two  key  enabhng  technologies  for  Global  Grid  were  Asynchronous  Transfer  Mode  (ATM) 
switches  and  satelhte  communications.  ATM  had  become  a  de-facto  standard  for  switches  to 
which  future  high  speed  networks  would  be  built  to.  Tactical  ATM  switches  were  being  built  by 
AFRL/fF’s  Secure  Survivable  Communications  Network  (SSCN)  Program  that  would  be  used 
in- theatre  during  mihtary  operations.  Satellite  communications  would  be  needed  to 
communicate  with  the  theatre  based  “crystal  island”  ATM  networks.  At  the  time,  the  space 
segment  of  this  architecture  was  undecided  and  conceivably  could  have  included  either 
geosynchronous  (GEO)  or  low  earth  orbit  (LEO)  satellites  and  may  have  used  crosslinks 
between  satellites. 

Given  this  general  objective,  four  tasks  were  defined.  The  first  was  to  perform  a 
network  connectivity  analysis  using  computer  simulations  to  identify  satellite  network 
architectures  that  would  support  the  Global  Grid  concept.  Options  were  to  include  Super  High 
Erequency  (SHE)  and  Extremely  High  Erequency  (EHE)  satellite  resources  in  GEO  or  LEO 
orbits,  with  and  without  crosshnk  capabilities.  The  second  task  would  have  provided  link 
analyses  for  the  up/downhnks  associated  with  the  recommended  satelhte  network  architectures. 
The  third  task  was  to  investigate  the  effects  of  the  relatively  high  bit  error  rates  of  satelhte 
communications  hnks  on  ATM  network  operations.  The  final  task  was  to  investigate  the  effects 
of  the  relatively  high  propagation  delays  of  satehite  communications  hnks  on  ATM  network 
operations. 


3.2  CONNECTIVITY  ANALYSIS 

Global  Grid  scenarios  were  set  up  on  our  MSS  End-To-End  Simulator  and  some  long¬ 
term  simulation  mns  were  made  for  connectivity  analysis.  Eor  these  scenarios,  the  ground 
segment  consisted  of  terminals  at  the  five  node  sites  planned  for  the  SSCN  Testbed  (USAE 
AERL/IE,  Rome  NY;  US  Army  CECOM,  Et.  Monmouth  NJ;  US  Navy  Nrad,  San  Diego 
CA;  USAE  ACC  Langley  AEB  VA;  and  DISA,  Et  Huachuca  AZ)  as  weh  as  several  other 
terminals  located  in  potential  theatre  locations.  As  a  basehne  space  segment,  nine  different 
satelhte  constehations  would  be  considered.  The  nine  constehations  chosen  had  been  the 
subject  of  investigation  of  a  previous  AERL/IF  in-house  program.  The  constehation  selection 
process  was  described  in  detail  in  the  final  report  for  that  program  and  wh  not  be  repeated 
here.  However,  the  constellations  themselves  are  summarized  below  in  Eigure  3. 
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Figure  3  -  Baseline  Candidate  Satellite  Constellations 


The  simulations  that  were  mn  didn’t  result  in  any  surprises.  Theoretically,  each  of  the 
sateUite  constellations  used  should  provide  100%  global  coverage.  The  configurations  labeled 
as  “basehne”  in  Figure  3  were  originally  generated  by  a  closed  form  solution  for  full  earth 
coverage  found  in  open  hterature.  The  remainder  of  the  configurations  was  variations  of  the 
three  “baseline”  configurations,  introducing  redundancy  through  doubting  the  number  of  satellites 
in  the  minimal  “baseline”  versions.  As  expected,  each  of  the  sateUite  constellations  that  were 
simulated  provided  very  nearly  continuous  coverage  at  each  of  the  earth  terminals  over  the 
length  of  time  simulated. 


3.3  LINK  ANALYSIS 

The  goal  of  this  task  was  to  do  link  budget  analyses  for  EHF,  SHF,  and  UHF  versions 
of  the  satellite  constellations  studied  and  determine  the  communications  hardware  requirements 
to  support  the  associated  links.  It  was  during  this  task  that  the  NATE  Validation  task  was 
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added  to  this  program.  As  a  result,  the  link  analysis  task  was  put  on  hold  while  the  NATE 
Vahdation  took  priority. 

Since  work  on  the  project’s  ATM  related  tasks  was  preempted  for  several  months  so 
that  the  NATE  Vahdation  could  be  accomplished,  it  was  appropriate  to  review  the  ATM  tasks 
to  determine  whether  it  stiU  made  sense  to  do  them.  By  that  time,  there  was  no  more  talk  of 
conceptual  space  architectures  containing  sateUites  in  both  geosynchronous  and  low  earth  orbit. 
Instead,  the  Global  Grid  would,  in  ah  likelihood,  rely  on  existing  satehite  communications 
resources  in  geosynchronous  orbit.  Therefore,  it  seemed  pointless  to  continue  to  calculate  link 
budgets  for  the  conceptual  satehite  constellations  described  above  and  this  task  was  effectively 
canceled.  The  remainder  of  the  connectivity  analysis  task  was  canceled  for  the  same  reason. 


3.4  BER  EFFECT  ANAL  YSIS 

This  task  was  started  with  a  literature  search  to  see  what  had  been  done  in  this  area 
already  and  what  this  program  might  be  able  to  add.  The  result  of  this  hterature  search  was  that 
quite  a  bit  had  already  been  done  to  study  the  effects  of  the  relatively  high  bit  error  rates 
associated  with  satehite  links  on  ATM  systems.  I  found  several  excehent  articles  and  decided 
that  I  could  not  contribute  any  original  work  of  any  significance  to  this  study  area.  The 
reference  section  of  this  report  wih  provide  references  to  work  in  this  area. 


3.5  PROP  AG  A  TION  DELA  Y  EFFECT  ANAL  YSIS 

This  analysis  was  an  expansion  of  an  analysis  documented  in  an  article  found  in  the  April 
1992  issue  of  IEEE  Communications  magazine.  The  article  addressed  the  fundamental 
relationship  between  latency  and  bandwidth  of  a  communications  hnk.  The  premise  of  this 
fundamental  relationship  is  that  increased  bandwidth  wih  only  equate  to  decreased  message 
transmit  times  if  the  queuing  plus  transmission  time  delay  is  greater  than  the  propagation  delay  of 
the  channel.  Eor  every  communications  hnk,  there  is  a  critical  bandwidth  beyond  which 
additional  bandwidth  no  longer  decreases  transmit  time  and  the  transmission  time  becomes 
limited  by  the  propagation  delay,  or  latency,  of  the  channel.  That  critical  bandwidth  is  defined 
as  fohows: 

1000  b 
“  (l-p)x 

hr  this  equation,  b  is  message  length  in  bits,  p  represents  the  system  load,  and  x  is  the 
propagation  delay  in  milliseconds. 

In  the  case  of  a  local  area  network,  propagation  delay  is  very  low  and  this  critical 
bandwidth  is  extremely  high,  making  the  use  of  very  wide  bandwidth  technologies  a  viable 
option.  Take  for  example  a  local  area  network  where  two  network  nodes  are  located  .25  miles 
apart.  This  distance  translates  to  a  propagation  delay  of  approximately  1.3  milliseconds  and  if 
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we  transmit  a  1  megabit  message  across  this  distance,  we  would  be  able  to  transmit  at  a  rate  of 
at  least  (0  load  case)  745  Gbps  before  our  link  became  latency  limited. 

If  we  consider  a  cross-country  link  from  New  York  to  Los  Angeles  instead,  the 
propagation  delay  is  much  higher,  but  we  can  stiU  take  advantage  of  wide  bandwidth 
technologies.  The  distance  from  New  York  to  Los  Angeles  is  approximately  2500  miles,  which 
translates  to  a  propagation  delay  of  13  milliseconds.  If  we  transmit  a  1  megabit  message  across 
this  distance,  we  would  be  able  to  transmit  at  a  rate  of  at  least  (0  load  case)  77  Mbps  before 
our  hnk  became  latency  limited. 

Finally,  we  consider  a  sateUite  hnk.  Of  the  most  commonly  used  cr  planned  sateUite 
orbit  altitudes,  the  most  common  and  worst  case  is  the  geosynchronous  orbit.  At  an  altitude  of 
22,284  miles,  a  geosyncronous  satelhte’s  altitude  is  roughly  an  order  of  magnitude  larger  than 
the  distance  from  New  York  to  Los  Angeles.  Furthermore,  since  both  an  uplink  and  a 
downlink  would  be  required  at  a  minimum,  the  distance  between  two  points  on  the  earth  via  a 
sateUite  would  be  roughly  twice  the  altitude  of  the  sateUite.  The  commonly  used  number  for 
propagation  delay  from  earth  to  sateUite  to  earth  is  250  milliseconds.  If  we  transmit  a  1  megabit 
message  across  this  distance,  we  would  be  able  to  transmit  at  a  rate  of  only  4  Mbps  before  our 
links  became  latency  limited. 

However,  the  4  Mbps  limitation  stated  above  is  but  one  example  of  the  critical 
bandwidth  in  a  satelUte  system.  It  corresponded  to  the  worst  case  geosynchronous  orbit  and  a 
zero  network  load  condition.  Also,  since  the  critical  bandwidth  is  a  function  of  message  size  as 
weU,  the  critical  bandwidth  wiU  increase  for  messages  larger  than  1  megabit. 

Figures  4  through  10  iUustrate  the  effects  of  satelUte  altitude,  network  load,  and  message 
size  on  the  critical  bandwidth  of  the  satelUte  system.  Figure  4  plots  critical  bandwidth  vs.  load 
as  a  function  of  satelUte  altitude.  Figure  5  plots  critical  bandwidth  vs.  load  for  a  1  MB  file 
relayed  through  a  variety  of  sateUite  consteUations  which  have  been  proposed,  and  in  some 
cases  developed,  in  recent  years.  A  generic  geosynchronous  altitude  sateUite  is  also  included. 
Figures  6- 10  plot  critical  bandwidth  vs.  load  for  a  variety  of  file  sizes  relayed  through  a  variety 
of  satelUte  consteUations. 
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CRITICAL  BW  VS.  LOAD  FOR  VARIOUS  SATELLITE  ALTITUDES  (in  Km) 
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Figure  4  -  Critical  Bandwidth  As  A  Function  Of  Altitude 


CRITICAL  BW  VS.  LOAD  FOR  VARIOUS  SATELLITE  ORBITS  (in  Km)  FOR  1  MB  FILES 
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Figure  5  -  Critical  Bandwidth  For  Selected  Satellite  Systems 
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CRITICAL  BW  VS.  LOAD  FOR  VARIOUS  FILE  SIZES  (in  KB) 
FOR  RELAY  THROUGH  GEO  ALTITUDE  SATELLITE 
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Figure  6  -  Critical  Bandwidth  As  A  Function  Of  File  Size  Through  GEO  Satellite 
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CRITICAL  BW  VS.  LOAD  FOR  VARIOUS  FILE  SIZES  (in  KB) 

FOR  RELAY  THROUGH  10,370  km  ALTITUDE  SATELLITE  (ODYSSEY) 
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Figure  7  -  Critical  Bandwidth  As  A  Function  Of  File  Size  Through  Odyssey  Satellite 
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CRITICAL  BW  VS.  LOAD  FOR  VARIOUS  FILE  SIZES  (in  KB) 

FOR  RELAY  THROUGH  10,370  km  ALTITUDE  SATELLITE  (INMARSAT  P) 
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Figure  8  -  Critical  Bandwidth  As  A  Function  Of  File  Size  Through  Inmarsat  P  Satellite 


CRITICAL  BW  VS.  LOAD  FOR  VARIOUS  FILE  SIZES  (in  KB) 

FOR  RELAY  THROUGH  10,370  km  ALTITUDE  SATELLITE  (GLOBALSTAR) 


Figure  9  -  Critical  Bandwidth  As  A  Function  Of  File  Size  Through  Globalstar  Satellite 
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CRITICAL  BW  VS.  LOAD  FOR  VARIOUS  FILE  SIZES  (in  KB) 

FOR  RELAY  THROUGH  1 0,370  km  ALTITUDE  SATELLITE  (IRIDIUM) 


Figure  10  -  Critical  Bandwidth  As  A  Function  Of  File  Size  Through  Iridium  Satellite 
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4.0  NATE  VALIDATION 


The  NATE  Validation  report  was  originally  prepared  by  Gregory  Hadynski  and  Capt. 
Michael  Mills  of  AFRL/IF.  While  there  have  been  no  changes  in  its  contents  for  inclusion  in  this 
report,  it  has  been  reformatted  to  form  an  integ'al  part  of  this  report. 

The  foreword  and  acknowledgments  for  the  NATE  Validation  section  of  this  report 
were  provided  by: 

Dr.  David  Einkleman,  SES-4 
Director  of  Analysis 

North  American  Aerospace  Defense  Command  and  United  States  Space  Command 

4.1  FOREWORD 

This  effort  is  a  milestone  in  the  new  rruhtary  modeling  and  simulation  environment  and  in 
joint  mihtary  efforts.  To  the  best  of  our  knowledge,  this  is  the  first  disciphned  vahdation  of  a 
composite  mihtary  model.  Our  commands  have  implemented  aggressively  guidance  from 
Congress  and  the  Office  of  the  Secretary  of  Defense.  We  are  building  substantiated  confidence 
in  the  models  and  simulations  that  we  use  for  planning,  training,  and  analysis.  AERL/IE  is  a 
knowledgeable,  independent  agency.  Its  capabihties  and  accomphshments  quahfy  the 
laboratory  to  examine  expertly  communications  models  and  computer  software. 
USSPACECOM  provided  the  software  and  sponsored  specific  training.  This  report  confirms 
the  value  of  our  relationship.  The  laboratory  found  deficiencies  in  the  software.  The  evaluators 
exposed  uncertainties  in  elements  of  the  software.  We  would  not  have  been  able  to  address 
these  matters  effectively  without  this  independent  vahdation.  By  the  time  we  distribute  this 
report,  we  should  have  replaced  the  outdated  missile  flyout  model  with  a  current  one  whose 
lineage  and  documentation  are  consistent  with  the  rest  of  the  simulation.  We  have  aheady 
remedied  minor  software  inconsistencies.  We  can  now  claim  much  greater  confidence  in 
analyses  that  use  this  tool.  At  this  writing,  we  have  engaged  AT&T  to  conduct  a  much  more 
comprehensive  vahdation  using  analysis  techniques  with  a  broad  commercial  basis.  This 
sequence  of  preliminary  ‘Tace  vahdation”  fohowed  by  more  comprehensive  formal  vahdation 
may  be  a  paradigm  for  the  future.  I  hope  that  this  report  enables  wider  implementation  of  the 
mandate  for  model  vahdation. 
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4.3  INTRODUCTION 

This  report  documents  AFRL/IF’s  recent  preliminary  validation  of  US  Space 
Command’s  NORAD  and  USSPACECOM  Integrated  Command  and  Control  (NUICCS) 
Analyst  Technical  Environment  (NATE).  This  work  was  performed  in  response  to  a  request 
made  by  US  Space  Command. 

The  vahdation  of  the  NATE  software  contributes  to  the  overall  objectives  of  US  Space 
Command’s  Project  EoreteU.  The  ultimate  goal  of  Project  EoreteU  is  to  develop  the  capabihty 
to  vahdate  rehably  the  potential  of  new  Air  Eorce  Satelhte  Control  Network  (AESCN)  system 
components  without  the  significant  expenditures  involved  in  developing  and  fielding  the  system 
components.  The  first  step  in  achieving  this  goal  is  to  assemble  a  modeling  and  simulation  test 
bed  by  connecting  and,  when  necessary,  vahdating  existing  models  and  simulations  of  space  and 
C4I  systems.  NATE  is  one  of  the  models  to  be  included  in  Space  Command’s  test  bed. 

There  were  four  team  members  involved  in  the  NATE  prehminary  vahdation.  AFRL/IF 
acted  as  the  team  lead,  an  independent  broker  hired  to  perform  an  objective  evaluation  of 
NATE.  US  Space  Command  was  the  customer,  providing  a  general  task  description  and 
operational  support.  Science  Apphcations  International  Corporation  was  the  developer  of  both 
NATE  and  one  of  the  principal  components  of  NATE,  the  Strategic  Command  &  Control 
Architecture  Model  (STRATC2AM).  They  provided  technical  support  on  the  subject  of  these 
two  models.  Einally,  the  Air  Eorce  Studies  and  Analysis  Agency  (AESAA),  the  government 
agency  responsible  for  the  development  of  STRATC2AM,  provided  vital  technical  support. 

This  report  is  basicahy  organized  according  to  the  main  tasks  involved  in  the  NATE 
validation.  The  first  task  was  to  plan  the  work  necessary  for  this  project.  The  second  task  was 
to  vahdate,  if  necessary,  the  individual  simulation  models  within  NATE.  The  third  task  was  to 
vahdate  the  interfaces  between  the  individual  models  within  NATE.  The  fourth  task  was  to 
vahdate  the  NATE  model  as  a  whole.  The  final  task  was  to  prepare  a  final  report.  An 
overview  of  the  NATE  simulation  environment  is  also  included  for  those  unfamihar  with  NATE. 
Appendix  A  contains  the  vahdation  plan.  It  was  included  as  an  appendix  rather  than  reiterating 
it  in  the  main  body  of  the  report.  Appendix  B  contains  a  hst  of  the  documents  referenced  to 
perform  this  analysis.  Appendices  C  &  D  contain  the  evaluation  matrices  which  document  the 
vahdation  results  and  form  the  basis  for  much  of  the  discussion  in  the  main  body  of  the  report. 

Since  this  report  wih  concentrate  on  NATE’s  problems,  it  is  important  to  put  the 
negative  comments  in  perspective.  While  NATE  may  have  its  problems  at  this  time,  it  has  many 
good  points  as  weh.  The  communications  codes  within  STRATC2AM  are  the  heart  of  NATE 
and  its  biggest  sehing  point.  It  is  in  communications  where  NATE’s  commercial  competitors  fah 
short.  Most  commercial  communications  network  programs  represent  free  space 
communications  hnks  simphsticahy,  requiring  the  user  to  provide  the  bit  error  rate  for  the  hnks. 
The  most  notable  exception  to  this  is  the  OPNET  model.  OPNET  allows  the  user  to  create 
their  own  link  models  in  code.  However,  STRATC2AM  has  a  robust  set  of  communications 
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link  codes  already.  Also,  NATE’s  Omni  graphical  user  interface  is  an  excellent  visualization 
tool,  providing  highly  detailed  animated  displays  of  global  C3I  scenarios. 


4.4  NATE  OVERVIEW 

NATE  has  a  storied  history  beginning  in  1974  when  work  began  on  the  models  which 
eventually  evolved  into  STRATC2AM,  one  of  the  four  main  components  of  NATE.  Today, 
NATE  consists  of  STRATC2AM,  the  Omni  graphical  user  interface,  the  COMET  missile  flyout 
model,  and  the  SDP4  &  SGP4  orbital  propagation  models. 

STRATC2AM  is  the  official  C3  model  for  AESAA  and  has  been  used  for  numerous 
apphcations  such  as  the  Strategic  C3  Systems  Review,  the  MIESTAR  Program  Review,  the 
NATO  C3  Architecture  Study,  the  Cheyenne  Mountain  Upgrade,  and  the  DSCS  SCT 
Upgrade.  It  supports  wide-area  network  simulations  with  space  nodes,  ground  nodes  and 
aircraft.  Each  node  model  includes  C2  node  processes  and  communications  transmission 
equipment  (EEE  through  optical).  Communications  between  nodes  are  modeled  for  benign 
environments,  jamming  environments,  and  nuclear  environments.  One  of  the  more  recent 
STRATC2AM  developments  has  been  the  addition  of  the  Analytical  X- Windows  Interface  to 
Simulations  (AXIS)  graphical  user  interface. 

The  development  of  the  NATE  environment  resulted  in  the  addition  of  a  second,  more 
powerful,  graphical  user  interface  known  as  Omni.  Omni  is  a  commercially  available  product 
sold  by  Autometric  Inc.  which  mns  on  a  Sihcon  Graphics  workstation.  Omni  supports  graphic 
data  analysis  and  provides  output  in  many  forms  including:  pictures,  graphs,  animated 
sequences,  and  text  windows.  It  does  this  using  a  mouse- controlled,  pull-down  menu-driven 
command  system  and  a  multiple,  overlapping  window  environment. 

The  remaining  two  NATE  components  are  actually  apphcation  software  for  the  Omni 
environment.  The  COMET  and  SDP4  &  SGP4  models  are  included  in  Omni’s  Astro 
application  package.  COMET  is  a  missile  propagation  model.  SDP4  &  SGP4  are  satellite 
propagation  models  originally  developed  for  NORAD.  SGP4  is  used  for  modeling  the  orbits  of 
near- Earth  satellites  and  SDP4  is  used  for  modeling  the  orbits  of  deep- space  satellites. 

A  block  diagram  depicting  NATE  operation  is  shown  in  Eigure  11. 
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4.5  PLANNING 

4.5.1  LEARNING  ABOUT  NATE 

While  learning  about  NATE  may  seem  like  an  obvious  step  in  its  validation,  its 
importance  really  does  warrant  inclusion  in  this  report.  Without  researching  a  model,  it  would 
be  impossible  to  develop  a  detailed  vahdation  plan.  Documenting  this  research  helps  to  vahdate 
the  validation  process. 

AFRL/lF’s  NATE  education  began  with  SAIC  visiting  AERL/IE  for  an  introductory 
briefing  and  demonstration.  This  visit  was  very  beneficial  and  left  AERL/IE  with  the  impression 
that  there  were  actually  many  similarities  between  NATE  and  models  which  AERL/IE  uses. 

Next,  AERL/IF  requested  and  received  full  sets  of  NATE  and  STRATC2AM 
documentation.  The  documentation  was  reviewed  by  the  engineers  who  would  be  working  on 
the  NATE  vahdation.  The  NATE  documentation  lacked  sufficient  information  on  the  COMET 
and  SGP4  &  SDP4  models.  Therefore,  additional  documentation  on  these  models  was 
requested.  The  requested  documentation  on  the  SGP4  &  SDP4  models  was  received,  but  for 
reasons  described  later  documentation  on  COMET  was  never  received.  A  full  list  of  the 
documentation  used  for  the  evaluation  is  included  in  Appendix  B. 

EinaUy,  the  AERL/IF  engineers  working  on  the  project  visited  SAIC  to  attend  a  NATE 
training  course. 
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4.5.2  FREEZING  OF  NATE  CONFIGURATION 

Early  in  the  planning  phase,  Spaee  Command  froze  the  NATE  eonfiguration  for  the 
duration  of  its  vahdation.  This  served  two  purposes.  Eirst,  it  told  AERL/IF  exaetly  what 
eonfiguration  it  would  be  using  for  its  analysis.  Seeond,  ehanges  to  the  NATE  eonfiguration 
during  its  validation  would  probably  require  the  vahdation  proeess  to  be  restarted  to  avoid 
suspeet  results.  The  NATE  eonfiguration  used  during  AERL/IE’s  analysis  eonsisted  of: 
STRATC2AM  C3  Model  (version  2.0) 

Omni  Graphical  User  Interface  (version  1.3.2) 

COMET  Missile  Elyout  Program 

SGP4  &  SDP4  Satellite  Propagation  Models 

NATE  Network  Architecture  (version  1.0) 


4.5.3  SEUECTION  OF  TEST  CASES 

The  selection  of  test  cases  for  the  vahdation  of  NATE’s  interfaces  was  an  arbitrary 
process  to  some  extent.  In  testing  the  interfaces,  the  main  concern  was  if  the  data  was 
transferred  across  the  interfaces  correctly,  and  not  if  the  data  itself  was  correct.  Eor  this  reason, 
we  used  an  exishng  scenario  called  “Simple.”  Simple  is  a  demo  which  was  created  by  SAIC  to 
demonstrate  the  network  switching  capabihty  of  NATE  using  protocol  mle  message  traffic 
routing. 

It  must  be  pointed  out  at  the  start  that  the  selection  of  particular  test  cases  for  the 
functional  vahdation  of  NATE  as  a  whole  is  vitahy  important.  The  test  case  to  be  used  needs  to 
be  representative  of  the  apphcation  which  Space  Command  ultimately  has  in  mind  for  NATE. 
Eor  this  reason.  Space  Command  chose  to  use  the  existing  “DSB”  database  which  describes  a 
strawman  surveihance  architecture  that  was  created  for  the  Defense  Science  Board.  The  DSB 
database  describes  a  fairly  large  scale  scenario  which  contains  global  communications  systems, 
national  C2I  resources,  national  information  gathering  assets,  theater  assets,  and  opposing 
forces. 


4.5.4  WRITING  NATE  VALIDATION  PLAN 

A  detailed  vahdation  plan  was  written  and  submitted  to  Space  Command  for  their 
approval.  The  plan  was  approved  with  minor  comments.  The  resulting  plan  is  included  in 
Appendix  A,  but  is  highhghted  here  for  convenience. 

The  plan  divides  the  NATE  vahdation  project  into  five  main  tasks:  planning,  submodel 
vahdation,  interface  vahdation,  NATE  validation,  and  prepare  final  report.  The  mechanics  for 
the  validation  process  were  adapted  from  the  “Analytical  Tool  Box  Eevel  1  Eace  Vahdation 
Assessment  Plan”  prepared  by  the  Martin  Marietta  Corporation  for  the  National  Test  Eacihty. 
Both  plans  use  a  combination  of  operational  effectiveness  evaluation  matrices  and 
corresponding  evaluation  criteria  to  rate  the  different  factors  of  a  simulation  model’s  operational 
effectiveness. 
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4.5.5  APPROVAL  OF  PLAN 


Timely  feedback  from  Space  Command  was  critical  to  the  success  of  this  project. 
Approval  of  the  plan  was  needed  before  the  validation  could  proceed  and  the  approval  needed 
to  be  expedited  to  avoid  unnecessary  schedule  impacts.  Since  the  evaluation  criteria  for  a 
software  model  vahdation  should  be  closely  tied  to  the  customer’s  intended  use  of  the  model, 
concurrence  with  Space  Command  was  very  important.  Fortunately,  the  plan  was  quickly 
accepted  with  a  few  minor  comments. 


4.6  SUBMODEL  VALIDATION 

Since  NATE  is  actually  a  composite  of  existing  models,  the  vahdation  plan  was  written 
as  a  methodology  for  vahdating  a  composite  model.  The  logical  approach  was  to  start  by 
vahdating  the  submodels.  However,  in  NATE’s  case,  the  submodels  had  undergone  vahdations 
in  the  past.  Eor  this  reason,  it  was  agreed  that  fuU  submodel  validations  did  not  need  to  be  done 
as  part  of  the  NATE  vahdation  and  that  submodel  vahdations  would  only  be  done  on  an  “as 
needed”  basis. 

Twice  during  this  effort,  the  need  for  vahdation  at  the  submodel  level  was  indicated.  In 
the  first,  testing  the  interface  between  the  STRATC2AM  post-processor  and  Omni  uncovered 
a  bug  which  was  eventually  determined  to  be  in  the  post-processor.  This  wih  be  described  in 
greater  detail  later  on  during  the  discussion  of  the  testing  of  that  interface. 

It  was  also  determined  that  a  fuh  vahdation  of  the  Comet  missile  propagator  submodel 
needed  to  be  done.  Unfortunately,  neither  the  time  nor  the  expertise  needed  to  do  this  was 
available  for  this  study.  Efforts  to  obtain  necessary  information  on  Comet  resulted  in  more 
questions  than  answers.  According  to  its  original  developers.  Comet  is  a  simphfied  missile 
propagation  code  which  has  been  around  for  many  years  and  provides  rehable  answers  to  the 
user  who  understands  the  limitations  of  the  model.  Its  most  recent  vahdation  took  place  in  late 
1989  and  early  1990,  when  a  1987-1988  version  was  examined  and  found  to  be  correctly 
coded.  However,  in  the  past,  various  users  of  Comet  have  informahy  made  independent 
changes  to  the  model  while  maintaining  the  original  program  name.  As  a  result,  there  are  several 
versions  of  Comet  in  existence,  only  one  of  which  is  official.  The  rest  of  the  versions  are 
unknown  quantities  to  Comet’s  developers.  Unfortunately,  the  version  of  “Comet”  used  in 
Omni  is  just  such  code.  Therefore,  Omni’s  version  rf  “Comet”  should  be  considered  an 
unknown  quantity  in  need  of  further  testing. 


4.7  INTERFACE  VALIDATION 

4.7.1  TESTS 

As  was  mentioned  earlier,  the  existing  NATE  scenario  “Simple”  was  used  for  the 
vahdation  of  NATE’s  interfaces.  In  Simple,  low  altitude  satehites  over  Korea  send  missile 
detection  messages  to  Colorado  Springs.  Initiahy,  these  messages  are  routed  through  a 


29 


constellation  of  Milstar  satellites.  However,  at  60  second  intervals,  the  low  altitude  satellites 
send  health  and  status  messages  through  the  Milstar  constellation  to  Colorado  Springs.  While 
the  health  and  status  messages  are  being  processed  by  Colorado  Springs,  the  Milstar  satellites 
are  not  available  for  the  routing  of  missile  detection  messages.  During  these  times,  the  missile 
detection  messages  are  instead  routed  through  an  SDS  constellation  to  Colorado  Springs.  The 
following  table  details  interfaces  that  were  tested: 


Omni  to  STRATC2AM 

Launched  node  entry 
Fixed  node  entry 
Moving  node  entry 
Satellite  entry 

Communications  control  file  entry 
Node  definition  file  entry 
STRATC2AM  to  Omni 

Communications  statistical  graphs 

Graph  time  delay  by  message  type 
Graph  link  message  count 
Graph  availability 
Graph  correct  message  receipt 
Graph  package  receipt 
Graph  resource  utilization 
Graph  link  loading 
Graph  link  utilization 
Graph  RF  performance 

Graph  probability  of  receipt 
Graph  signal  to  interference 
Graph  absorbance 
Graph  scintillation 
Graph  decorrelation 

Show  links 
Link  filters 
Link  shading 
Omni  to  SGP4  &  SDP4 

Satellite  orbit  descriptions 
SGP4  &  SDP4  to  Omni 

Satellite  positions 


Table  1:  NATE  Interfaces  Tested 
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4.7.2  RESULTS 


4.7.2.1  0MNI/SGP4  &  SDP4  INTERFACES 

The  first  interface  that  was  tested  was  the  interface  between  Omni  and  the  SGP4  and 
SDP4  orbital  propagation  models.  Project  Spacetrack  Report  No.  3  which  documents  the 
SGP4  and  SDP4  models  includes  examples  of  results  which  the  models  should  yield.  Using 
these  examples,  we  were  able  to  compare  the  results  provided  by  Omni  with  the  results  which 
the  models  provide.  In  doing  so,  we  discovered  that  Omni  doesn’t  allow  a  user  to  input  the 
orbital  parameters  to  the  same  degree  of  precision  which  the  standard  two- card  element  set 
description  uses.  The  mean  motion  loses  one  digit  in  the  process  of  inputting  the  parameters 
while  entering  the  epoch  results  in  losing  three  digits. 

Overall,  the  results  provided  by  Omni  agreed  fairly  well  with  the  results  contained  in  the 
Spacetrack  report,  but  they  did  not  match  perfectly.  While  some  error  should  be  expected  due 
to  different  computer  word  lengths  according  to  the  Spacetrack  report,  it  is  difficult  to  determine 
if  the  missing  digits  of  precision  were  solely  responsible  for  the  variance  without  adding  the 
missing  digits  and  rechecking  the  results.  If  the  versions  used  in  Omni  are  later  versions  than  the 
ones  described  in  the  Spacetrack  report,  then  that  could  account  for  some  of  the  variance  in 
results.  Attempting  to  provide  a  quantitative  measure  of  the  errors  in  the  Omni  results,  we 
determined  the  magnitude  of  the  position  vectors  from  the  xyz  coordinates  that  the  SDP4  and 
SGP4  models  provide,  and  compared  the  Omni  results  to  the  values  in  the  Spacetrack  report. 
For  the  near  earth  orbit  case,  the  magnitude  of  the  error  was  approximately  .02  kilometers.  For 
the  deep  space  orbit  case,  the  magnitude  of  the  error  was  approximately  1.3  kilometers. 

Since  there  is  no  apparent  reason  why  this  precision  mismatch  could  not  be  fixed,  we 
strongly  recommend  that  a  correction  be  made  to  the  Omni  data  input  window  to  accommodate 
the  required  precision. 


4.7.2.2  STRATC2AM  POSTPROCESSOR/OMNI  INTERFACE 

The  next  interface  which  was  tested  was  the  interface  between  the  STRATC2AM 
Post-processor  and  Omni.  Omni  creates  graphs  of  several  STRATC2AM  post-processing 
reports.  Omni  does  this  by  plotting  points  from  text  files  generated  by  the  STRATC2AM 
Postprocessor  for  each  of  the  statistical  reports.  Therefore,  testing  the  interface  was  simply  a 
matter  of  visually  comparing  the  text  files  generated  by  STRATC2AM  with  the  graphs  created 
by  Omni. 

Although  this  test  sounds  trivial  (since  it  isn’t  likely  that  Omni  can’t  plot  points 
correctly),  when  asked  to  graph  statistics  for  link  utilization,  link  load,  and  hnk  demand,  Omni 
provided  some  interesting  results.  The  graphs  actually  contained  a  loop.  Part  of  the  hnk 
utilization  graph  is  shown  in  Figure  12  to  iUustrate  this. 
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Figure  12:  Link  Statistics  Anomaly  Example 


Upon  further  examination,  we  determined  that  Omni  was  doing  exactly  what  it  was 
supposed  to  do.  It  plotted  the  points  which  STRATC2AM  provided.  The  loop  in  the  graphs 
was  caused  by  one  data  point  being  out  of  sequence  in  the  file  which  STRATC2AM  created. 
Therefore,  this  is  not  a  problem  with  the  interface,  but  with  STRATC2AM  itself.  Also,  it 
proved  to  be  an  intermittent  problem.  The  same  link  statistics  were  generated  for  several  other 
links  in  the  same  scenario.  None  of  them  exhibited  the  same  problem.  According  to  SAIC,  this 
is  a  problem  they  have  seen  before,  but  thought  had  been  corrected. 

Irregularities  were  also  discovered  in  plotting  the  postprocessor’s  Grouped  Packet 
Receipt  report.  The  first  irregularity  is  that  this  report  has  three  different  names  depending  on 
where  you  look.  STRATC2AM  generates  what  it  calls  a  Grouped  Packet  Receipt  report.  The 
Communications  Statistical  Graph  menu  in  Omni  calls  it  Package  Receipt,  and  the  graph  is 
labeled  Group  Packet  Probabihty  of  Receipt.  At  best,  this  is  a  minor  oversight.  At  worst,  this 
is  tmly  confusing. 

Unfortunately,  semantics  was  not  the  only  problem.  This  graph  did  not  seem  technically 
correct  either.  According  to  the  STRATC2AM  postprocessor  file,  with  only  one  data  point  in 
this  case,  the  Omni  output  graph  should  have  had  a  value  of  0  probabihty  of  receipt  until  t  = 
3.96  seconds,  at  which  time  the  graph  should  have  had  the  value  1.  When  the  graph  was 
checked,  this  isn’t  what  was  found.  First,  the  y-axis  scale  was  wrong.  Since  the  data  points 
were  probabihties  for  this  graph,  the  scale  should  range  from  0  to  1.  Instead  it  ranged  from  I  to 
a  value  that  was  off  the  graph  and  could  not  be  seen.  Next,  the  graph  began  at  3.96  seconds 
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and  there  was  no  value  shown  at  all  mtil  3.96004  seconds.  At  3.96004  seconds,  the  value 
shown  was  approximately  midway  between  1  and  the  maximum  value  on  the  y-axis. 

Finally,  Omni’s  use  of  continuous  hne  graphs  to  plot  some  of  STRATC2AM’s  outputs 
can  be  very  misleading.  For  example,  Omni  will  indicate  positive  link  utilizations  during  periods 
of  link  inactivity  simply  because  the  link  is  active  before  and  after  the  period  of  inactivity.  If  a 
hne  graph  is  going  to  be  used,  it  is  adviseable  that  the  individual  datapoints  be  highhghted.  In 
many  cases,  histograms  may  be  even  more  effective.  This  is  especiaUy  hue  for  the  link  message 
count  statistic.  The  data  which  STRATC2AM  provides  for  its  link  message  count  statistic  was 
meant  to  be  displayed  in  a  histogram  format,  but  Omni  did  not  have  a  histogram  capabihty  at  the 
time. 


4.7.2.3  OMNI/STRATC2AM  PREPROCESSOR  INTERFACE 

The  main  interface  between  Omni  and  the  STRATC2AM  preprocessor  is  through  what 
Omni  terms  a  “platform.”  Within  Omni,  the  user  defines  fixed  sites,  satehites,  moving  nodes, 
and  missiles.  A  “platform”  associates  these  fixed  sites,  satelhtes,  moving  nodes,  and  missiles 
with  the  communications  hardware  onboard.  From  there,  Omni  will  take  a  set  of  platforms  and 
either  create  a  new  STRATC2AM  node  definition  file  or  append  the  platforms  to  an  existing 
node  definition  file.  Omni  will  also  let  the  user  take  the  node  definition  file  that  they  just  created 
and  mn  STRATC2AM  without  leaving  Omni. 

This  helps  to  reduce  the  switching  between  apphcations  which  the  NATE  user  needs  to  do. 

We  created  one  fixed  site,  one  sateUite,  and  one  moving  node  in  Omni  and  tried  both 
creating  a  new  STRATC2AM  node  definition  file  with  these  platforms  and  appending  them  to 
an  existing  node  definition  file.  The  results  were  consistent  between  these  two  approaches.  In 
either  case,  the  values  of  some  of  the  position  variables  for  the  platforms  were  changed  shghtly 
by  the  interface  process.  However,  the  magnitude  of  these  errors  was  neghgible.  It  is  only 
mentioned  here  for  the  sake  of  completeness.  More  troubhng  is  the  fact  that  the  interface 
apparently  converted  the  moving  node  into  a  launched  node. 

Perhaps  the  most  significant  comment  that  can  be  made  about  the  interface  between 
Omni  and  the  STRATC2AM  preprocessor  is  that  preprocessor  input  through  Omni  is  very 
limited.  While  new  nodes  can  be  created  from  within  Omni,  virtually  everything  else  must  stiU  be 
done  in  the  STRATC2AM  preprocessor. 


4.7.2.4  OMNI/COMET  INTERFACE 

Although  testing  the  Omni/Comet  interface  was  part  of  the  original  vahdation  plan,  it 
was  decided  that  this  be  postponed.  It  is  recommended  that  the  Comet  code  in  Omni  be 
vahdated  as  a  submodel  prior  to  testing  this  interface.  Neither  the  time  nor  the  expertise  to 
accomphsh  this  task  was  available  for  this  study. 
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4.7.2.5  STRATC2AM  PREPROCESSOR  &  POSTPROCESSOR/USER 
INTERFACE 

STRATC2AM’s  pre-processor  and  post-processor  are  both  adequate  functionally,  but 
are  definitely  not  user-friendly.  The  pre- processor  is  a  command- line  user  interface  which 
makes  data  input  a  very  long  process.  In  fact,  the  interface  is  so  difficult  to  use  that  some  users 
actually  prefer  to  use  a  text  editor  to  create  and  modify  preprocessor  files  rather  than  use  the 
pre- processor.  Also,  inconsistencies  in  the  user  interface  can  be  very  fmstrating.  Different  keys 
must  be  pressed  for  the  same  actions  depending  on  where  the  user  is  in  the  interface.  The  good 
news  is  that  this  should  be  much  less  of  a  problem  very  soon.  The  next  version  of 
STRATC2AM  is  to  sport  a  new  graphical  user  interface  which  looks  to  be  a  quantum  leap  for 
the  STRATC2AM  user.  Nearly  everything  that  currently  must  be  entered  by  way  of  the 
command- line  interface  wiU  be  able  to  be  entered  using  simple  point  and  chck  instmctions.  The 
exceptions  to  this  are  jammers,  events,  and  user- defined  modems  which  stiU  wiU  require  the  old 
pre-processor. 

The  post- processor  is  not  quite  as  difficult  to  deal  with  as  the  current  pre- processor,  but 
it  is  StiU  in  need  of  updating.  The  user  request  reports  by  using  a  menu/command- line  interface. 
The  reports  themselves  are  tabular  data  available  in  several  formats  for  plotting  in  other  software 
programs  (e.g.  Lotus  Freelance,  Omni,  etc.).  The  post- processor  should  be  modernized  to  use 
a  graphical  interface  and  create  presentation  quaUty  tables  and  graphs.  The  metrics  themselves 
should  also  be  revisited.  Some  metrics  may  no  longer  be  useful  and  others  could  be  added. 
For  example,  a  useful  metric  to  add  would  be  connectivity  (within  tine -of- sight  within  range) 

as  a  function  of  time.  There  is  currently  a  report  available  which  shows  nodes  within  Une-of- 
sight  as  a  function  of  time,  but  this  doesn’t  teU  the  whole  story. 


4.8  NATE  VALIDATION 

4.8.1  TESTS 

NATE,  including  the  DSB  database,  was  evaluated  using  the  evaluation  criteria  detailed 
in  the  vaUdation  plan  for  Space  C3  Planning  as  a  yardstick.  DSB  describes  an  attack  by  North 
Korean  forces  on  South  Korea.  The  North  Korean  headquarters  signals  the  pre- positioned 
tank  battaUons  to  move  toward  the  South  Korean  border.  In  concert  with  the  arrival  of  the 
North  Korean  forces  at  the  border,  a  SCUD  attack  is  initiated.  During  the  initial  phase  of  the 
scenario,  the  North  Korean  headquarters  signals  the  tank  battahons  and  the  national  signal 
intelligence  assets  intercept  the  transmission  which  is  then  forwarded  to  the  data  processing  and 
C2  nodes  in  CONUS.  The  interception  of  this  information  triggers  data  being  sent  back  to  the 
theater  Air  Operations  Center  to  schedule  the  launch  of  theater  based  assets.  Concurrently,  as 
national  electro- optic  and  synthetic  aperture  radar  satelhtes  make  passes  over  the  area,  data  is 
collected  and  forwarded  to  CONUS  for  data  processing  and  as  inputs  to  the  C2  nodes.  This 
information  is  then  forwarded  to  the  Air  Operations  Center.  Once  the  theater  airborne  assets 
reach  operational  altitudes,  the  information  they  gather  is  sent  back  to  the  Control  and  Reporting 
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Center.  When  the  SCUD  is  launched  from  North  Korea,  it  is  detected  by  the  DSP  satellites. 
These  satelhtes  forward  the  data  to  the  CONUS  Missile  Warning  Center.  The  CONUS 
Missile  Warning  Center  sends  data  to  the  theater  Air  Operations  Center  which  forwards  the 
data  to  the  Patriot  battery. 


4.8.2  RESULTS 

From  a  communications  perspective,  the  NATE  model  framework  (not  including  the 
database)  is  acceptable  for  Space  Command’s  intended  purpose  with  a  couple  of  small 
exceptions.  First,  the  data  rate  cannot  be  set  by  the  user  for  hardwired  links  currently.  Second, 
there  is  a  bug  in  the  STRATC2AM  software  which  causes  datapoints  to  be  recorded  out  of 
sequence  in  some  postprocessor  reports,  resulting  in  errant  statistical  plots. 

Of  course,  there  is  always  room  for  improvement  in  any  model  and  NATE’s 
communications  model  is  no  exception.  While  it  is  not  absolutely  necessary,  NATE  would  also 
benefit  from  adding  more  modem  link  types  (e.g.  ethemet,  FDDI,  ATM).  Clever  users  can 
probably  emulate  these  types  of  links  with  the  current  NATE,  but  it  would  require  the  user  to  be 
a  communications  expert.  Similarly,  modeling  of  techniques  for  multiple  access  to  resources 
such  as  FDMA,  TDMA,  and  CDMA  would  be  useful.  Modehng  antenna  position  and  slewing 
would  increase  reahsm  for  sateUite  and  other  hne- of- sight  communications.  Currently,  the 
assumption  is  made  that  antennas  are  always  pointing  in  the  right  direction.  At  a  minimum,  the 
user  should  be  able  to  choose  some  nominal  antenna  slewing  time.  Also,  since  modem  mihtary 
communications  satelhtes  such  as  DSCS  rely  on  MBAs  for  beam  steering,  beam  shaping,  and 
jamming  suppression,  an  MBA  model  would  be  a  good  addition. 

Unfortunately,  when  we  define  NATE  to  include  the  DSB  database,  the  model  is  less 
than  acceptable.  In  a  way,  it  doesn’t  seem  fair  to  criticize  the  DSB  database,  because  it  was 
not  developed  with  reahsm  in  mind.  It  was  designed  to  provide  an  unclassified  demonstration  of 
NATE’s  capabihties  and  it  does  that.  However,  simulation  results  are  only  as  good  as  the  data 
that  a  user  puts  into  a  model  and  Space  Command  needs  to  know  that  simulation  scenarios 
using  the  DSB  database  will  not  give  them  reahstic  results.  In  fact,  the  DSB  database  would  not 
provide  results  at  ah  unless  the  scenario  is  mn  using  the  unclassified  development  version  of 
NATE  which  ignores  communications  link  calculations.  When  this  was  discovered,  SAIC 
modified  the  database  so  that  it  would  at  least  provide  some  results  when  mn  on  the  official 
version  of  NATE.  While  the  DSB  database  may  be  a  fair  representation  of  the  types  of 
resources  that  wih  be  available  in  future  conflicts  (Milstar,  DSCS,  imaging  satelhtes.  Patriot 
batteries,  etc.)  the  resources  are  not  accurately  described  in  the  database.  Without  discussing 
exact  specifications  of  the  operational  systems,  we  can  stih  give  a  few  examples.  The  operating 
frequencies  defined  for  the  Milstar  transmitters  are  off  by  anywhere  from  40  to  70  GHz.  This 
has  very  real  imphcations  in  trying  to  predict  the  attenuations  on  uphnks  and  downlinks  due  to 
the  atmosphere  and  rain.  In  general,  the  bandwidths  specified  are  quite  wide.  In  fact,  one  of 
the  TDRSS  satelhte  transmitters  has  a  100  GHz  bandwidth  while  the  operating  frequency  is  only 
7.5  GHz.  This  is  worse  than  inaccurate.  This  is  physicahy  impossible  and  NATE  should  give 
error  messages  for  mistakes  of  this  nature. 
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Again,  the  main  point  of  these  examples  is  to  stress  that  one  of  Space  Command’s  top 
priorities  regarding  NATE  needs  to  be  to  develop  a  realistic  database  of  present  and  projected 
resources  to  base  their  scenarios  on.  Unfortunately,  this  is  currently  a  non- trivial  task. 
However,  the  graphical  user  interface  for  the  preprocessor  with  the  next  version  of 
STRATC2AM  should  make  this  a  much  more  manageable  process. 

The  environment  is  adequately  modeled  for  Space  Command’s  purposes  at  this  time. 
Environmental  effects  modeled  presently  are  atmospheric  loss,  rain  loss,  jamming,  and  nuclear 
burst  effects.  There  has  also  been  talk  recently  of  ‘tacticahzing’  STRATC2AM  to  make  it  more 
apphcable  to  tactical  scenarios.  While  it  is  probably  not  necessary  for  Space  Command’s 
purposes,  terrain  modeling  would  greatly  benefit  the  tactical  users  of  STRATC2AM. 

Erom  Space  Command’s  perspective,  another  area  where  NATE  is  currently  lacking  is 
in  sensors.  NATE  does  not  currently  model  sensors.  Sensors  can  be  “faked”  to  some  extent 
by  using  Omni.  Omni  allows  the  user  to  attach  a  very  simphfied  model  of  a  sensor  to  any 
platform.  It  allows  the  user  to  display  a  cone  with  a  user  specified  field  of  regard  to  emulate  a 
sensor.  However,  this  is  only  a  visual  tool.  If  a  missile  flies  directly  through  a  sensor’s  field  of 
regard,  nothing  will  happen.  There  is  no  connection  between  Omni  and  STRATC2AM 
allowing  the  generation  of  missile  warning  messages  triggered  by  such  an  event.  Currently,  this 
is  a  manual  process.  The  user  visually  detects  a  sensor  event,  notes  the  time,  and  modifies  the 
STRATC2AM  preprocessor  file  to  add  missile  warning  messages  at  that  time.  At  a  minimum, 
this  process  could  be  automated.  If  more  realism  is  desired,  a  link  could  be  made  to  an  existing 
sensor  model  such  as  the  Mission  Effectiveness  Model  (MEM).  Whatever  form  it  takes,  a  fuUy 
functional  sensor  model  should  be  a  fairly  high  priority  for  NATE. 

Threat  resolution  and  discrimination  could  currently  be  emulated  by  the  NATE  user 
similar  to  way  that  sensors  can  be  emulated,  by  a  manual  process.  The  main  difference  would 
be  that  the  process  would  be  even  clumsier.  Since  messages  in  NATE  do  not  contain  any  data, 
different  message  types  would  need  to  be  created  to  indicate  threat  resolution  and  discrimination 
results.  The  effort  that  would  be  required  for  NATE  to  actually  model  resolution  and 
discrimination  would  not  be  trivial,  and  unless  it  is  a  critical  issue  to  Space  Command,  it  would 
probably  not  be  the  best  use  of  scarce  NATE  development  dollars. 

Tracking  and  data  fusion  cannot  be  done  currently  with  NATE  and  the  effort  associated 
with  adding  this  capabihty  would  be  considerable.  Algorithms  for  performing  the  tracking  and 
data  fusion  would  obviously  need  to  be  added.  However,  there  is  stiU  the  underlying  limitation 
of  NATE  that  messages  do  not  currently  contain  data.  Eortunately,  this  is  probably  not  a  high 
priority  item. 

Surveillance  and  inteUigence  are  also  not  currently  included  in  NATE,  and  in  aU 
likelihood  they  never  will  be.  Adding  inteUigence  data  to  NATE  would  add  not  one  but  two 
levels  of  complexity  to  the  model.  Not  only  would  it  be  necessary  to  pass  data  in  messages,  it 
would  be  necessary  to  judge  whether  or  not  the  data  was  genuine.  Since  this  is  probably 
beyond  the  scope  of  Space  Command’s  intended  use  for  NATE,  it  was  probably  unfair  to 
include  it  as  an  evaluation  factor.  On  the  other  hand,  it  is  fair  because  it  would  be  part  of  the 
scenario  in  real  Ufe. 

It’s  notable  to  point  out  that  we’ve  just  mentioned  three  things  in  a  row  which  NATE 
cannot  reaUy  do  presently  due  partiaUy  to  its  inabiUty  to  pass  data  in  messages.  Therefore,  the 
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addition  of  the  capability  of  passing,  interpreting,  and  acting  on  data  could  potentially  have  very 
widespread  benefits.  Despite  the  fact  that  STRATC2AM  stands  for  STRATegic  Command 
and  Control  Architecture  Model,  it  is  primarily  a  communications  model.  Passing  data  instead 
of  bits  in  messages,  NATE  could  become  a  true  C3  model. 

Some  aspects  of  electronic  warfare  (EW)  are  addressed  fairly  well  while  others  are  not 
addressed  at  ah.  Eor  scenarios  which  only  include  the  jamming  of  communications  hnks, 
NATE’s  EW  model  is  adequate.  Since  the  jammer  antenna  is  assumed  to  always  be  pointing  in 
the  right  direction,  modeling  jammer  antenna  pointing  would  provide  added  reahsm.  However, 
the  argument  could  be  made  that  simphfications  resulting  in  a  worst  case  situation  is  not 
necessarily  a  bad  thing.  Unfortunately,  countermeasures  associated  with  sensors  are  not 
addressed  at  ah.  Of  course,  since  NATE  does  not  really  have  a  sensor  model,  this  is 
understandable. 

Random  processes  are  handled  acceptably  in  NATE  with  the  exception  of  determining 
equipment  availabihty.  It’s  good  that  the  rehability  of  equipment  is  addressed,  but  it’s 
implemented  in  an  awkward  manner.  Probabihty  of  failure  is  specified  for  entire  classes  of 
equipment  instead  of  individual  pieces  of  equipment.  What  this  means  is  that  every  piece  of 
equipment  having  the  same  class  wiU  fah  at  the  exact  same  time.  Since  this  is  unlikely  to  ever 
happen  in  real  fife,  it  seems  more  reahstic  to  ignore  probabihty  of  equipment  failure  altogether. 
The  only  way  to  make  this  feature  work  properly  is  for  the  user  to  not  use  more  than  one  of  any 
one  class  of  equipment,  thereby  not  taking  advantage  of  the  convenience  of  being  able  to  define 
classes.  This  should  be  fixed. 


4.9  CONCLUSIONS  &  RECOMMENDATIONS 

The  stated  purpose  of  this  project  was  not  to  do  an  in-depth  vahdation  of  NATE’s 
individual  models.  It  was  given  that  these  models  had  been  sufficiently  vahdated  already.  Our 
prime  objective  was  to  determine  whether  any  of  the  individual  models  had  been  “broken”  in  the 
process  of  integrating  them  together.  This  was  done  through  extensive  testing  of  the  interfaces 
between  the  individual  NATE  components.  The  results  of  these  tests  showed  that  the  individual 
models  had  not  been  “broken”,  although  they  were  “chipped”  in  a  few  places.  These  “chips” 
can  be  found  below  under  the  heading  of  EKES.  We  have  also  provided  a  frank  assessment 
of  NATE’s  suitabihty  to  Space  Command’s  intended  use. 

While  this  report  is  full  of  NATE’s  problems,  NATE  is  basically  a  very  sound 
communications  simulation  model.  This  would  be  a  much  longer  document  if  we  also  included 
everything  in  NATE  that  was  right.  Almost  aU  of  the  mistakes  which  were  found  in  the  design 
appear  to  be  relatively  minor  and  definitely  fixable.  Since  a  few  of  the  mistakes  were  in  the 
commercial  product  Omni,  the  biggest  problems  with  fixing  them  may  be  contractual  ones.  Its 
most  obvious  weak  point  is  a  very  old-fashioned  and  user- unfriendly  pre-processor  user 
interface  and  this  will  be  nearly  eliminated  in  the  next  release  of  STRATC2AM  (due  for  release 
any  time  now).  Its  next  biggest  weak  point  is  that  reahstic  databases  need  to  be  built  up  to  base 
simulation  scenarios  on.  The  reason  why  this  is  such  a  problem  currently  is  that  the  existing  pre¬ 
processor  would  make  this  a  very  time-consuming  task  best  left  to  STRATC2AM  experts.  If 
the  new  graphical  pre- processor  is  as  good  as  it  appears  to  be,  this  wiU  be  a  much  more 
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manageable  task  that  could  be  done  by  the  users  instead  of  the  developer.  For  the  intended  use 
of  NATE  the  next  highest  priority  change  should  be  the  addition  of  a  real  sensor  model.  At  that 
point,  NATE  should  be  fairly  well  suited  for  Space  Command’s  purposes. 

Of  course,  there  will  always  be  something  that  a  user  will  want  to  add  to  any  model  and 
NATE  is  no  exception.  Below  is  a  list  of  enhancements  which  we  would  currently  recommend 
for  NATE.  Beyond  the  suggestions  to  build  realistic  databases  and  add  a  sensor  model  which 
have  already  been  discussed,  we  do  not  recommend  them  in  any  particular  order.  Also,  as  was 
mentioned  above,  a  hst  of  recommended  fixes  is  included.  Since  making  a  fix  to  a  model  means 
that  there  is  something  incorrect  in  the  model  which  could  be  compromising  simulation  results, 
the  fixes  should  be  considered  higher  priority  items. 


4.9.1  FIXES 

1.  The  Omni  sateUite  data  entry  windows  should  be  fixed  to  accommodate  the  precision 
required  for  standard  two-card  orbit  descriptions. 

2.  STRATC2AM  pre- processor  should  be  fixed  to  accept  user- specified  data  rates  for 
hardwired  finks. 

3.  STRATC2AM  post- processor  should  be  fixed  to  ensure  that  time  sequencing  of  data 
points  is  correct. 

4.  The  irregularities  in  the  pbtting  Grouped  Packet  Receipt  report  in  Omni  should  be  fixed. 

5 .  Continuous  fine  graphs  in  Omni  should  be  fixed  to  highlight  the  data  points. 

6.  The  error  in  Omni  that  causes  moving  nodes  to  be  interpreted  as  launched  nodes  when 
written  to  STRATC2AM  node  definition  files  should  be  fixed. 

7.  The  equipment  probability  of  failure  model  should  be  fixed  to  allow  the  independent 
failures  of  members  of  an  equipment  class. 

4.9.2  ENHANCEMENTS 

1 .  Develop  a  realistic  database  that  can  provide  system  “building  blocks”  that  users  can 
use  with  confidence  to  build  scenarios  with. 

2.  Add  real  sensor  capabilities  to  NATE. 

3.  Develop  a  new  STRATC2AM  post- processor  with  a  point- and- click  type  interface. 

4.  Add  modem  fink  types  (e.g.  ethemet,  EDDI,  ATM,  etc.) 

5.  Validate  the  “Comet”  missile  flyout  model  in  Omni  to  determine  if  it  conforms  to  the  real 
Comet  model.  If  it  does,  validate  Omni’s  interface  to  Comet. 

6.  Complete  the  graphical  user  interface  pre- processor  which  the  next  version  of 
STRATC2AM  will  use. 

7.  If  possible,  create  unclassified  “development”  version  of  STRATC2AM  which  does  not 
ignore  fink  calculations.  Perhaps  this  could  be  done  by  eliminating  the  nuclear  codes 
and  only  using  generic  modems  (e.g.  DPSK,  CESK,  NESK,  and  CPSK). 

8.  Add  connectivity  statistics  metric. 

9.  Add  resource  multiple  access  models  (e.g.  EDMA,  TDMA,  and  CDMA). 


38 


1 0 .  Add  MBA  model. 

1 1 .  Add  capability  to  pass,  interpret,  and  act  on  data  in  messages. 

4.9.3  DRAFT  AIR  FORCE  INSTRUCTION  16-1002 

DRAFT  AIR  FORCE  INSTRUCTION  16-1002  defines  validation  as  a  rigorous  and 
stmctured  process  of  determining  the  extent  to  which  an  M&S  accurately  represents  the 
intended  "real  world"  phenomena  from  the  perspective  M&S  use,  and  may  identify 
improvements.  It  has  two  main  components:  stmctural  vahdation,  which  includes  an  internal 
examination  of  M&S  assumptions,  architecture,  and  algorithms  in  the  context  of  the  intended 
use;  and  output  vahdation,  which  determines  how  weU  the  M&S  results  compare  with  the 
perceived  "real  world." 

Although  the  instmction  was  not  approved  at  the  time  of  the  NATE  validation,  we  feel  it 
is  important  to  comment  on  the  degree  to  which  this  vahdation  addressed  the  above  definition. 
The  instmction  further  specifies  the  fohowing  vahdation  methodologies  that  could  be  commented 
on.  Upon  signature  of  the  instmction,  these  comments  would  allow  reference  of  the  report  to 
the  instmction. 

Eace  vahdation  process  that  determines  whether  an  M&S,  on  the  surface,  seems  reasonable  to 
personnel  knowledgeable  about  the  system  being  modeled;  (addressed  by  this  report) 

Comparison  with  historical  data  or  with  results  from  other  M&S  aheady  accredited  for  use  in 
similar  applications;  (not  addressed) 

Comparison  with  developmental  test  data  or  other  engineering  test  data;  (to  some  extent) 

Comparison  with  operational  test  data,  other  field  tests,  or  operational  data;  (Appendix  A  - 
NATE  Vahdation  and  Accreditation  Plan  requested  Desert  Shield  Data  Bases  and  the  DSP 
Missile  Warning  Dissemination  data,  however  the  only  test  data  available  was  the  Defense 
Science  Board  database  and  the  "simple"  scenario.) 

Peer  review,  where  functional  area  SMEs  analyze  M&S  internal  representations  and  outiouts; 
(accomphshed  to  some  extent  by  AFRE/IE’s  review) 

Independent  review  of  the  entire  M&S,  or  specific  functions,  by  a  designated  committee  or 
other  agents  that  are  independent  of  the  M&S  developer,  (this  was  the  accomphshed  by 
AERE/IF’s  review) 


4.9.4  FUTURE  NATE  CONFIGURATIONS  AND  VAUIDATION 

Now  that  this  preliminary  vahdation  of  NATE  has  been  completed,  it  is  a  fair  question 
to  ask  what  happens  when  NATE’s  configuration  changes.  Wih  this  vahdation  be  worthless 
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then?  Will  it  need  to  be  completely  redone?  That  would  depend  upon  how  much  NATE’s 
configuration  changes.  Obviously,  if  aU  of  NATE’s  components  are  replaced,  then  the  answer 
is  yes.  However,  if  a  component  is  being  added,  then  the  answer  is  probably  no. 

Eor  example,  if  a  sensor  model  is  added  as  has  been  suggested,  NATE’s  vahdation 
would  need  to  be  updated,  but  would  not  need  to  be  completely  redone.  Eirst,  the  sensor 
model  would  be  vahdated  if  it  has  not  already  been  validated.  Second,  the  interfaces  between 
the  sensor  model  and  NATE’s  other  components  would  be  tested.  It  should  be  noted  that  this 
may  not  be  a  trivial  step.  Ironically,  the  interfaces  for  the  current  configuration  were  fairly  easy 
to  test  due  to  the  fact  that  STRATC2AM  is  stiU  old-fashioned  in  many  ways.  The  interface 
from  the  post-processor  to  Omni  was  via  a  textual  file  containing  a  hst  of  data  points.  If  this 
interface  was  by  a  more  direct  means,  it  could  have  been  much  more  difficult  to  test.  EinaUy, 
the  new  NATE  would  be  evaluated  against  the  Space  C3  Planning  criteria  used  for  this 
vahdation.  The  evaluation  of  the  new  NATE  would  be  done  by  updating  the  appropriate  parts 
of  the  old  evaluation. 

If  STRATC2AM  were  to  be  replaced  by  a  different  model,  the  bulk  of  this  vahdation 
would  need  to  be  redone.  STRATC2AM  is  the  heart  of  NATE  and  the  effects  of  its 
replacement  on  NATE’s  vahdation  would  be  widespread.  The  new  communications  model 
would  need  to  be  vahdated  if  it  had  not  already  been  vahdated.  Most  of  the  interfaces  would  be 
changed,  requiring  new  testing.  Einahy,  much  of  the  functional  evaluation  (against  the  Space  C3 
Planning  criteria)  would  need  to  be  redone  to  reflect  the  new  communications  model. 
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4. 10  APPENDIX  A -NATE  VALIDA  TION  AND  ACCREDIT  A  TION  PLAN 


The  NATE  Validation  and  Accreditation  Plan  was  originally  prepared  by  Gregory 
Hadynski  and  William  Brennan  of  AFRL/IF.  While  there  have  been  no  changes  in  its  contents 
for  inclusion  in  this  report,  it  has  been  reformatted  to  form  an  integral  part  of  this  report. 


4.10.1  OVERVIEW 

United  States  Space  Command  has  recently  initiated  a  program  to  perform  a  vahdation 
and  accreditation  of  4ie  NUICCS  Analyst  Technical  Environment  (NATE)  simulation  model. 
The  formal  vahdation  and  accreditation  wiU  be  performed  by  AT&T  under  contract  to 
USSPACECOM  with  funding  by  the  Defense  Modeling  and  Simulation  Office  (DMSO). 
However,  by  USSPACECOM  request,  AFRL/IF  wiU  work  with  the  Air  Force  Center  for 
Studies  and  Analysis  Agency  (AFSAA)  and  Science  Apphcations  International  Corporation 
(SAIC)  to  provide  an  informal  vahdation  of  the  NATE  model  in  the  interim.  AFSAA  is  the 
agency  responsible  for  the  development  of  one  of  the  principal  parts  of  the  NATE  model,  the 
Strategic  Command  and  Control  Architecture  (STRATC2AM)  model.  AFSAA  and  SAIC 
have  performed  vahdation  of  the  STRATC2AM  model  as  an  integral  part  of  its  development 
cycle  and  wiU  ensure  that  the  NATE  vahdation  goes  hand-in-hand  with  their  efforts.  This  plan  is 
an  AFRL/IF  document  and  wih  concentrate  on  the  AFRL/AFSAA/SAIC  informal  vahdation 
effort. 


4.10.2  STRATEGY 

The  NATE  vahdation  and  accreditation  initiative  is  especiaUy  chaUenging  in  that  NATE 
itself  is  a  composite  of  simulation  models.  NATE  consists  of  four  main  components:  the 
STRATC2AM  C3  simulator,  the  Omni  graphic  interface,  the  COMET  missile  propagator,  and 
the  SGP4  &  SDP4  satehite  propagators. 

It  is  logical  to  begin  the  vahdation  process  of  a  composite  model  by  decomposing  the 
model  to  its  major  components  and  performing  a  vahdation  process  on  each  of  the  major 
components.  Eortunately,  the  individual  models  that  NATE  consists  of  have  already  been 
sufficiently  vahdated  to  eliminate  much  of  this  step.  STRATC2AM,  COMET,  and  SGP4  & 
SDP4  have  ah  stood  up  weh  to  scmtiny  over  a  long  period  of  time.  A  recent  memorandum 
from  Curt  Smith  of  SAIC  to  Lt.  Col.  Keith  James  of  USSPACECOM  provides  an  excehent 
overview  of  the  history  of  STRATC2AM  validation.  The  STRATC2AM  Analyst’s  Manual 
provides  actual  validation  data.  COMET  and  SGP4  &  SDP4  have  been  used  by  NORAD  for 
many  years.  Since  Omni  is  a  graphic  interface  rather  than  a  model,  it  cannot  reahy  be  vahdated 
alone.  Eor  these  reasons,  the  individual  models  within  NATE  wih  be  vahdated  under  this  effort 
on  an  “as  needed”  basis  determined  by  vahdation  results  of  the  NATE  model  as  a  whole. 

Having  done  this,  we  need  to  vahdate  the  “glue”  which  holds  these  models  together. 
Again  we  can  decompose  the  problem  by  individually  addressing  the  interfaces  between  each  of 
the  models.  Our  goal  in  this  step  is  to  ensure  that  vahdated  data  generated  by  any  one  of  the 
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individual  models  is  accurately  transferred  to  the  appropriate  model.  This  step  is  essentially 
checking  data  at  the  beginning  (databases  and  pre-processors),  midpoint,  and  endpoint  (GUI 
and  post- processors)  in  the  NATE  environment  and  provides  greater  confidence  in  the  eventual 
output  of  NATE.  While  all  of  the  interfaces  to  be  tested  are  internal  to  NATE,  most  of  them 
should  be  relatively  simple  to  test.  The  majority  of  the  interfaces  are  between  STRATC2AM 
and  NATE  and  testing  these  interfaces  would  entail  a  straightforward  comparison  of  the  outputs 
of  STRATC2AM  to  the  outputs  of  NATE. 

The  last  step  is  to  vahdate  the  NATE  model  as  a  whole.  This  would  be  done  by  using 
simulation  scenarios  of  complexity  adequate  for  testing  NATE’s  main  functions.  These 
scenarios  will  be  provided  by  USSPACECOM,  but  could  include  the  detection  of  ballistic 
missiles  and  the  dissemination  of  threat  warnings.  Desert  Storm  scenarios,  or  perhaps  everyday 
conditions.  While  “real  world  data”  for  communications  systems  is  never  absolutely  repeatable 
due  to  the  probabihstic  nature  of  communications,  our  goal  is  to  determine  levels  of  confidence 
that  can  be  put  in  NATE  results.  NATE  will  be  evaluated  against  an  extensive  set  of  criteria  to 
determine  these  levels  of  confidence. 


4.10.3  VALIDATION  PROCESS 

Eor  guidance  on  the  vahdation  process,  the  AERL/IF  vahdation  team  will  look  to  Dr. 
Paul  Davis’s  report  on  Generahzing  Concepts  and  Methods  of  VV&A  for  Military  Simulations 
as  well  as  the  most  current  Air  Eorce  guidance  and  practical  experience.  The  figure  shown 
below  is  taken  from  Dr.  Davis’s  report  and  illustrates  the  vahdation  process  which  wiU  be 
generally  foUowed.  Using  this  philosophy,  a  combination  of  empirical  evaluation,  theoretical 
evaluation,  and  other  comparisons  are  used  to  determine  levels  of  confidence  in  the  model. 


Additional  guidance  for  AERL/IF  in  performing  Vahdation  and  Accreditation  of  NATE 
can  be  found  in  Eigure  2  Taxonomic  View  of  VV&A  (found  in  the  same  report  mentioned 
above).  The  blocks  of  the  diagram  which  are  applicable  to  this  validation  program  are 
highhghted.  The  branch  of  this  tree  of  importance  to  AERL/IF's  vahdation  effort  is  "Generahzed 
Vahdation  (Evaluation)".  Ah  three  branches  under  "Generalized  Vahdation  (Evaluation)"  are  of 
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interest  to  some  extent.  NATE  results  wiU  be  evaluated  against  “real  world”  data  where 
possible.  Where  “real  world”  data  is  not  available,  results  will  be  evaluated  against  results  from 
other  simulations  or  against  theory.  Finally,  engineering  judgement  may  be  used  to  determine 
the  reasonableness  of  results. 

This  methodology  of  vahdation  agrees  with  the  direction  of  USSPACECOM  to  assure 
that  NATE  represents  the  physical  and  operational  processes  it  was  developed  for,  through 
comparison  of  "one  dimensional"  results  from  NATE  to  those  of  the  independent  models.  For 
example,  does  STRATC2AM  give  the  same  answers  alone  as  it  does  when  incorporated  in 
NATE?  This  also  addresses  the  vahdation  of  the  Graphical  User  Interfaces  (OMNI  and  AXIS) 
in  providing  complete  and  accurate  data  presentation. 

NATE  wiU  be  vahdated  through  careful  comparison  of  selected  "real  world"  cases. 
Independent  test  data  obtained  from  the  Desert  Shield  Data  Bases  and  the  DSP  Missile 
Warning  Dissemination  are  two  example  test  cases  to  be  used  for  the  empirical  evaluation  of 
NATE.  Assessment  of  selected  NATE  output  data  to  determine  physical  accuracy  and 
soundness  wiU  be  accomphshed  with  particular  attention  given  to  communications  delays, 
dropout,  and  surveiUance  coverage  as  a  function  of  time. 
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Figure  14  -  A  Taxonomic  View  ofW&A 
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The  mechanics  of  the  validation  process  will  follow  the  process  described  in  the 
Analytic  Tool  Box  (ATB)  Level  1  Face  Vahdity  Assessment  Plan  developed  by  Martin 
Marietta  Corporation  for  the  National  Test  Facility  (NTF)  at  Falcon  Air  Force  Base.  While  the 
Face  Validity  Assessment  Plan  was  written  as  a  means  to  evaluate  a  simulation  model  for 
acceptance  into  NTF’s  Analytic  Tool  Box,  it’s  process  is  quite  applicable  to  our  task  of 
evaluating  the  NATE  simulation  model  for  acceptance  into  USSPACECOM’s  planned 
modeling  and  simulation  testbed.  More  specifically,  for  the  NATE  preliminary  validation  effort, 
the  use  of  face  validity  assessment  evaluation  matrices  will  be  adopted. 


4.10.4  FACE  VALIDITY  ASSESSMENT  EVALUATION  MATRICES 

The  assessment  process  is  made  repeatable  through  the  use  of  a  set  of  evaluation 
matrices  and  checklists.  These  hsts  capture  the  primary  factors  that  contribute  to  the  M&S 
quality  and  operational  effectiveness  as  well  as  chart  the  progress  made  toward  vahdation.  Each 
primary  factor  or  area  is  further  decomposed  into  subfactors.  The  subfactors  themselves  are 
evaluated  by  means  of  a  set  of  criteria  and  exploratory  questions  that  either  probe  the 
information  or  directly  lead  to  evaluative  answers.  While  many  areas  of  the  matrix  are  quite 
exphcit,  others  are  somewhat  more  difficult  to  assess  and  may  require  further  refinement  by  the 
team.  These  subfactors  are  evaluated  by  the  team  through  a  consensus  process  and  then 
combined  into  an  overall  evaluation.  This  consensus  approach  significandy  reduces  the 
subjectivity  of  the  assessment. 

A  non- numerical  rating  system  with  four  possible  scores  ("Exceeds  Guidelines,"  "Meets 
Guidelines,"  "Minor  problems  Noted,"  and  "Major  Problems  Noted")  is  used  to  rate  the  primary 
factors  and  subfactors  for  operational  effectiveness.  The  principal  objection  to  using  a  numerical 
scoring/weighting  system  to  combine  all  of  the  ratings  into  a  single  “overall  score”  is  that  an 
“overall  score”  does  not  capture  the  variations  in  the  importance  and  completeness  of  the 
evaluation  that  can  be  performed  in  the  separate  factors  and  therefore  may  present  a  misleading 
picture  of  results  of  the  process. 

The  evaluation  material  is  partitioned  into  two  different  matrices,  with  each  including  and 
evaluation  criteria: 

a.  Operational  Effectiveness- Integration/Data  management 

b.  Operational  Effectiveness- Space  C3  Planning  (Not  a  standard  matrix  in  the  TB 
Pace  Validity  Assessment  Plan.  Created  for  the  USSPACECOM  mission) 


4.10.5  TASK  DESCRIPTIONS 


Task  1: 

A  plan  for  performing  the  informal  vahdation  of  the  NATE  model  wiU  be  developed. 
The  document  which  you  are  reading  constitutes  the  most  current  version  of  the  plan.  It  wiU  be 
revised  throughout  the  effort  as  required.  The  final  version  of  this  plan  wiU  be  incorporated. 
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along  with  the  vahdation  results,  into  an  AFRL/IF  Technical  Report  upon  completion  of  the 
effort. 

Included  in  this  task  will  be  AFRL/IF  efforts  to  learn  the  NATE  system  and  its 
associated  components. 

Feedback  from  US  Space  Command  is  critical  for  the  completion  of  this  task  and  the 
consequent  execution  of  the  remaining  tasks.  Approval  of  the  plan  must  be  expedited  to  avoid 
schedule  impacts.  Since  the  evaluation  criteria  should  be  closely  tied  to  the  customer’s  intended 
use  of  the  model,  concurrence  with  Space  Command  is  very  important.  Scenarios  of  interest  to 
Space  Command  should  also  be  identified  during  the  planning  period. 


Task  2: 

Individual  models  which  constitute  the  NATE  model  will  be  vahdated  as  required. 
Since  the  individual  models  have  already  undergone  considerable  vahdation,  activity  in  this  task 
is  expected  to  be  hmited. 

Task  3: 

The  interfaces  which  bind  the  individual  NATE  models  wiU  be  vahdated.  The  NATE 
interfaces  which  wih  be  vahdated  are  hsted  in  Table  1.  The  evaluation  matrix  and  accompanying 
evaluation  criteria  for  this  task  can  be  found  in  Appendix  A. 
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MAIN  NATE  INTERFACES: 

Omni  to  STRATC2AM 
Missile  entry 
Platform  entry 
Sensor  entry 
Site  entry 

Moving  object  entry 
Communications  control  file  entry 
Node  definition  file  entry 
STRATC2AM  to  Omni 

Communications  statistical  graphs 

Graph  time  delay  by  message  type 
Graph  link  message  count 
Graph  availability 
Graph  correct  message  receipt 
Graph  package  receipt 
Graph  resource  utilization 
Graph  link  loading 
Graph  link  utilization 
Graph  RE  performance 

Graph  probability  of  receipt 
Graph  signal  to  interference 
Graph  absorbance 
Graph  scintillation 
Graph  decorrelation 

Show  links 
Link  filters 
Link  shading 
Omni  to  COMET 

Missile  records 
COMET  to  Omni 

Missile  positions 
Omni  to  SGP4  &  SDP4 

Ephemeris  descriptions  of  satellites 
SGP4  &  SDP4  to  Omni 

Satellite  positions 


Table  1.  Main  NATE  Interfaces 


Task  4: 

A  validation  of  the  NATE  model  as  a  whole  will  be  performed.  The  exact  scenarios  to 
be  simulated  are  TBD  at  this  time.  They  will  be  provided  by  USSPACECOM.  The  evaluation 
matrix  and  accompanying  evaluation  criteria  can  be  found  in  Appendix  B. 

Task  5: 

An  AERL/IF  Technical  Report  will  be  written  and  pubhshed  which  will  document  the 
vahdation  plan,  test  results,  and  quantitative  levels  of  confidence  in  the  NATE  model. 
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Schedule: 


Task 


1994 


Oct  I  Nov  I  Dec  |  Jan  |  Feb  |  Mar  |  Apr  |  May  |  Jun  Jul  |  Aug  |  Sep 


Oct  Nov  Dec  Jan  Feb 


1  Planning 

1 .1  Study  Model 

1 .2  Write  Draft  Pian 

1 .3  Updates  of  Pian 

1.4  Freeze  Configuratio 

1 .5  Seiect  Test  Cases 

1 .6  Approval  of  Plan 

1 .7  Recieve  Scenarios 

2  Submodel  Validation 

3  Interface  Validation 

3.1  Assemble  Referencf 

3.2  Run  Test  Cases 

3.3  Examine  Logic 

4  NATE  Validation 

4.1  Assemble  Referencf 

4.2  Run  Test  Cases 

4.3  Analysis/Compariso 

5  Prepare  Final  Report 


r 


□ 

c 


□ 


□ 


Figure  15  -  NATE  Validation  Schedule 
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4.10.7  APPENDIX  A  -  EVALUATION  MATERIAL  FOR  OPERATIONAL 
EFFECTIVENESS-INTEGRATION/DATA  MANAGEMENT 


OPERATIONAL  EFFECTIVENESS  EVALUATION  MATRIX  - 
INTEGRATION/DATA  MANAGEMENT _ 


Figure  16  -  Integration/Data  Management  Evaluation  Matrix 
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4.10.7.1  EVALUATION  CRITERIA  FOR  OPERATIONAL 
EFFECTIVENESS  -  INTEGRATION/  DATA  MANAGEMENT 


1.  MODEL  COMPATIBILITY.  Lor  an  integrated  phenomenology  model  framework,  the 
various  constituent  M&S  should  be  able  to  represent  the  phenomenologies  at  the  level  of 
detailed  resolution  expected  in  order  to  be  used  as  intended..  The  M&S,  when  integrated 
together  to  form  a  composite  solution,  should  have  data  interfaces  and  source  data  commonahty 
that  are  compatible  in  accuracy  and  functional  use.  If  one  model  provides  macro  level  results 
while  another  provides  micro  level  results,  one  cannot  expect  to  achieve  micro  level  results 
when  used  in  composite.  The  integrated  model  framework  should  also  provide  compatibihty 
across  the  family  of  constituent  M&S  with  regard  to  assumptions  and  limiting  constraints.  If 
assumptions  are  in  conflict,  or  limitations  of  one  model  are  being  violated  by  the  use  of  another 
model,  then  the  results  of  model  integration  have  less  value.  It  should  be  possible  through  model 
demonstration,  interviews  with  developers  and  IV&V  personnel,  and  review  of  documentation 
to  obtain  a  reasonably  good  subjective  understanding  of  the  level  of  model  compatibihty  within 
the  integrated  model  framework.  Lor  the  model  to  "meet  guidehnes"  the  integration  of  the 
constituent  M&S  must  be  compatible  in  assumptions,  limitations/constraints,  use  of  physical 
parameters,  and  there  must  be  consistency  between  model  and  real  object  fidelity. 

a.  Assumptions 

How  compatible  are  the  model  assumptions  with  respect  to  the  frinctionahty, 
performance,  and  intended  environment  being  modeled? 

How  adequate  are  assumptions  regarding  intended  use  and  accuracy  of  model 
results? 

How  adequate  are  assumptions  regarding  use  and  accuracy  of  source  data? 

How  adequate  are  assumptions  regarding  model  integration  to  achieve  composite 
results? 

How  adequate  are  assumptions  regarding  the  model  representation  fidehty  of 
phenomenologies? 

b.  Limitations 

How  compatible  are  the  model  limitations  with  respect  to  the  frinctionahty, 
performance,  and  intended  environment  being  modeled? 

How  clearly  are  the  limitations  stated? 

What  impact  do  the  limitations  have  on  use  of  models  for  critical  decision¬ 
making? 

What  are  the  conditions  under  which  limitations  apply? 

What  is  the  level  of  fidehty  between  phenomenology  and  models? 

c.  Physical  Parameters 

How  compatible  are  the  physical  parameters  with  the  assumptions,  limitations,  and 
phenomenologies  of  the  models  being  used? 


49 


What  is  the  relationship  of  physieal  parameters  to  phenomenology  models? 
What  is  the  souree  of  physieal  parameter  values? 

How  is  the  integration  of  physieal  parameters  aeeomphshed  aeross  various 
models? 


d.  Fidehty 

How  tmsted  is  the  model  fidelity-  the  modeling  results  eompared  with  aetual 
phenomenologies  being  modeled? 

Is  there  similarity  of  results  where  fiinetional  redundaney  exists  aeross  models? 

Was  there  suffieient  eertifieation/vahdation  of  model  souree  data? 

Was  there  suffieient  eertifieation/vahdation  of  model  algorithms? 

Is  there  eonsisteney  of  model  results  with  measured  phenomenology  data? 

2.  INTEGRATION.  For  an  integrated  phenomenology  model  framework,  the  various 
eonstituent  M&S  must  be  adequately  integrated  through  model  to  model  interfaees  and  model 
to  framework  interfaees.  The  integration  depends  upon  the  data  used  by  the  M&S  (souree-  and 
seenario-  speeifie),  shared  among  M&S,  and  eolleeted  from  model  exeeution  for  exeeution 
display  and  post- exeeution  analysis.  The  integrated  model  framework  should  be  reviewed  to 
ensure  an  appropriate  level  of  detail  is  provided  by  eaeh  of  the  M&S  as  integrated  within  the 
framework.  The  doeumentation  should  expheitly  deseiibe  all  interfaees  between  the  integrated 
model  framework  and  external  sourees,  between  any  one  model  and  the  integrated  model 
framework,  and  between  eooperating  M&S.  Demonstrations  should  illustrate  how  M&S  are 
integrated  together  to  aehieve  desired  results.  For  the  model  to  "meet  guidehnes"  there  must  be 
suffieient  model/framework  seene  generation  interfaees,  model/model  interfaees  for  operational 
use,  user  interfaees  for  input  and  output  of  information,  and  supporter  interfaees  for 
aeeomplishing  model  modifleations. 

a.  Model/Framework  Interfaees 

How  adequate  are  the  models  (e.g.,  phenomenology-  terrain,  eloud,  horizon, 
earthlimb,  aurora,  spaee,  nuelear,  boost,  mideourse,  vent,  debns)  within 
the  integrated  model  framework  (e.g.,  seene  generation-  framework  seenario 
speeifieations,  transformation,  eonstmetion  eomponent,  framework 
phenomenology  model  data  hbraries  eomponent,  framework  model  exeeution 
eomponent,  framework  seene  generation  eomponent)? 

Is  there  eonsisteney  in  the  level  of  detail  for  model/framework  interfaees? 

Is  there  eonsistent  data  fidelity  and  souree  of  data? 

Are  there  suffieient  modes  of  operation  (e.g.,  seene  generation-  models  and  flames, 
models  only,  frames  only,  information)  that  affeet  whieh  interfaees  are  used  and 
the  fidehty  of  the  results? 

b.  Model/Model  Interfaees 

How  adequate  are  the  interfaees  between  models  or  modules? 

Is  there  eonsisteney  in  the  level  of  detail  for  model/model  interfaees? 
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Is  there  consistent  data  fidelity  and  source  of  data? 

What  capabilities  are  provided  for  model  data  (input,  output)  to  be  used  by  other 
models  without  excessive  assumptions,  constraints,  and  transformations? 

c.  User  Interface/Input  &  Output 

How  adequate  are  the  user  interfaces  for  accomphshing  an  efficient 
scenario  constmction,  model  execution,  and  report  generation? 

How  adequate  is  the  scenario  generation  interface  provided  the  human  user? 

How  adequate  is  the  framework  and  its  model  components  to  efficiently  handle  data 
input  during  execution  of  the  models? 

How  adequate  is  the  framework  for  entry  and  modification  of  various  types  of  data 
including:  physical  data  constants,  threat  data,  phenomenology  data? 

How  adequate  are  the  display  interfaces:  pre- execution,  execution,  and  post¬ 
execution? 

How  adequate  is  the  framework  in  providing  a  user  interface  to  specify  the  form 
and  content  of  model  results  for  use  in  analysis? 

d.  Runtime  Data  Management 

How  adequate  is  the  management  of  data  (input,  display,  and  output)  during  the 
execution  of  the  selected  set  of  models  within  the  integrated  model 
framework? 

How  efficient  is  the  data  management;  e.g.,  does  it  take  a  long  time  to  enter  data? 

Does  it  take  a  long  time  to  process  data? 

Does  it  use  excessive  machine  and  human  resources  to  complete  its  modeling 
function? 

Is  the  information  presented  during  execution  simple  and  understandable  to  the 
human  operator? 

Is  there  sufficient  consistency  in  the  information  presented  for  input,  display,  or 
output? 

Is  there  capabihty  provided  for  storage  of  data  for  later  use  during  analysis? 

e.  Model  Modification  Interfaces 

How  adequately  does  the  model  support  modifications  to  its  component 
elements? 

Is  there  allowance  for  modifications  to  physical  parameters  (constants)? 

Is  there  allowance  for  modifying  accuracy  levels  in  results? 

Is  there  sufficient  means  provided  for  modifications  to  enhance  models? 

Is  there  sufficient  means  provided  for  modifications  to  correct  defects? 

Is  there  sufficient  means  provided  for  modifications  to  adapt  to  changes  in 
threat,  environment,  or  other  model/framework  interfaces? 

Is  there  sufficient  means  provided  for  modifications  to  integrate  with  new  models? 
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3.  DATA  MANAGEMENT.  Eor  an  integrated  phenomenology  model  framework,  the  various 
constituent  M&S  should  consistently  handle  the  management  of  data  flowing  into  and  out  of  the 
integration  framework.  These  individual  M&S  may  not  have  originally  been  designed  without 
such  integration  considered,  so  it  is  imperative  that  upon  integration  the  data  management  be 
baselined  and  forrealized.  The  efficient  management  of  the  data  through  the  use  of 
preprocessors,  scenario  generators,  report  generators,  off-the-shelf  analysis  tools,  and 
databases  can  aid  in  providing  a  synergistic  data  management  capabUity.  This  synergism 
depends  upon  how  weU  common  data  requirements  have  been  identified,  how  weU  the  data  is 
prepared  prior  to  model  execution,  the  mechanisms  available  during  execution  to-  capture 
sigpificant  data  necessary  for  other  model  use  and/or  post- execution  analysis,  the  analysis 
processes  used  to  derive  information  from  the  data  collected,  and  the  presentation' 
representations  of  the  data  to  capture  the  significance  of  the  modehng  results.  Models  without 
the  mechanisms  to  adequately  manage  input,  nantime  internal,  nantime  display,  and  output  data 
wiU  not  be  able  to  adequately  support  the  required  technical  decision-making  process.  Eor  the 
model  to  "meet  guidelines"  it  must  adequately  use  procedures  for  collecting,  storing,  displaying, 
and  analyzing  data  during  the  preparation,  mntime  execution,  and  postexecution  processing 
activities  related  to  the  integrated  model  framework. 

a.  Data  Preparation 

How  adequate  are  the  provisions  in  the  process  for  preparing  data  used  by 
the  integrated  model  framework  and  its  constituent  models? 

Are  input  data  matrices  sufficient? 

Is  there  a  user  interface  for  building  the  appropriate  input  data  files? 

Are  there  adequate  preprocessors,  such  as  scenario  constmction  tools,  scenario 
transformation  tools,  and  certified  data  sources? 

b.  Data  Collection 

How  adequate  is  the  process  for  collecting  data  used  by  the  integrated  model 
framework  and  its  models  for  inputs  and  post- execution  analysis? 

Are  there  automated  data  collection  instmmentation  within  the  models? 

Is  there  provision  of  controls  for  selecting  and  limiting  the  data  collected  and  the 
form  of  the  data  collected? 

c.  Data  Processing/Analysis 

How  adequate  is  the  processing  and  analysis  of  data  collected  during  the 
execution  of  the  integrated  model  framework  and  its  models? 

How  well  have  off-the-shelf  tools  been  integrated  into  the  data  processing/analysis 
process? 

Is  there  sufficient  control  of  the  processing/analysis  methods  to  properly  reflect 
desired  fidehty  of  results? 

How  adequate  is  the  use  of  processing/analysis  tools  during  pre- execution, 
execution,  and  post- execution? 
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d.  Data  Presentation 

How  adequate  is  the  process  of  presenting  data  during  execution  (e.g.,  data 
displays)  and  post- execution  analysis? 

Are  there  sufficient  graphical  capabihties  during  preparation,  execution,  and 
analysis? 

Is  there  capability  to  provide  levels  of  data  presentation  from  summary  to  very 
detailed  depending  upon  the  audience? 

Is  there  means  for  automated  integration  of  model  execution,  data  collection,  and 
data  presentation? 
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4.10.8  APPENDIX  B  -  EVALUATION  MATERIAL  FOR  OPERATIONAL 
EFFECTIVENESS-SPACE  C3  PLANNING 


OPERATIONAL  EFFECTIVENESS  EVALUATION  MATRIX  - 
SPACE  C3  PLANNING _ 


Figure  17  -  Space  C3  Planning  Evaluation  Matrix 
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4.10.8.1  EVALUATION  CRITERIA  FOR  OPERATIONAL 
EFFECTIVENESS  for  SPACE  C3  PLANNING 


These  criteria  are  intended  for  rating  medium-fidelity  architecture  models. 

1.  FIDELITY 

a.  Communications 

Communications  in  the  model  should  have  realistically  described  equipment  and  comm 
processes  which  are  of  medium  fidelity  as  noted  below.  In  addition,  the  comm 
processes  must  be  integrated  with  the  other  processes  (i.e.,  affect  and  are  affected  by 
them).  An  overall  score  of  "exceeds"  guidelines  requires  that  the  simulation  model  the 
type,  timing,  and  failure  of  aU  messages  (data)  as  a  function  of  capacity,  traffic,  routing, 
and  environment.  An  overall  score  of  "meets"  guidelines  require  that  the  model 
represents  the  explicit  transfer  of  information  for  key  decisions  with  delays  and 
connectivity  explicitly  modeled.  An  overall  score  of  "minor  problems  noted"  should  be 
used  when  the  model  represents  communication  delays,  but  with  some  deficiencies, 
inconsistencies,  omissions,  or  incomplete  integration  with  other  processes.  An  overall 
score  of  "major  problems  noted"  should  be  used  only  connectivity  is  modeled  or 
communications  are  not  integrated  with  other  processes. 

b.  Electronic  Warfare 

Electronic  Warfare  (EW)  is  an  increasingly  important  aspect  of  air  warfare.  Effects  of 
countermeasures  should  be  explicitly  included.  An  overall  score  of  "exceeds"  guidelines 
requires  that  the  simulation  explicitly  model  sensors,  comm,  and  weapons  for  both  sides, 
reactively.  Sensor  and  communications  degradation  is  based  on  accurate  geometric 
modeling  of  interference  (jamming  and  self-jamming).  An  overall  score  of  "meets" 
guidelines  require  that  the  model  represents  major  effects  of  countermeasures  to  key 
sensors,  comm,  and  weapons  in  a  balanced  fashion.  Uses  variables  and  mechanisms 
with  due  regard  to  physical  law.  An  overall  score  of  "minor  problems  noted"  should  be 
used  when  the  model  represents  some  effects,  but  (for  example)  in  scripted  form, 
without  explicit  modeling  of  physical  effects.  An  overall  score  of  "major  problems 
noted"  should  be  used  ff  EW  is  one-sided  only  or  is  not  included. 

c.  Environment 

Environmental  factors  should  be  included  in  a  medium- fidelity  simulation.  These  factors 
should  include  phenomenology  which  affect  communications  links.  An  overall  score  of 
"exceeds"  guidelines  requires  that  the  simulation  model  such  phenomenology  as: 
atmospheric  attenuation,  attenuation  due  to  rain,  jamming,  nuclear  burst  effects,  dust 
storms,  smoke,  clouds,  wind,  temperature,  vegetation,  man-made  terrain  features,  and 
dynamic  terrain.  The  simulation  should  also  include  environmental  effects  on  ground 
equipment,  degradation  of  sensors,  and  weapon  probability- of- kill  (Pk).  Terrain 
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features,  both  man-made  and  natural  should  be  well  represented.  An  overall  score  of 
"meets"  guidelines  require  that  atmospheric  attenuation,  attenuation  due  to  rain,  jamming, 
and  nuclear  burst  effects  be  accurately  modeled.  An  overall  score  of  "minor  problems 
noted"  should  be  used  when  the  model  ignores  some  of  the  effects  of  the  environment 
on  communications  links.  An  overall  score  of  "major  problems  noted"  should  be  used  if 
the  effects  of  the  environment  are  ignored  or  incorrect. 

d.  Detection 

Detection  is  a  necessary  component  of  a  meaningful  engagement  model.  Model 
capabihties  should  consider  sensors  and  their  supporting  systems,  and  interface  them 
effectively  into  support  of  other  systems  functions.  An  overall  score  of  "exceeds" 
guidelines  requires  that  the  simulation  model  accurately  model  all  sensor  types' 
performance  at  varying  range  in  the  presence  of  natural  and  threat  environments,  with 
due  consideration  to  sensor  capabihties,  field  of  regard,  field  of  view,  and  sensor  slew 
rates.  Sensor  modehng  should  consider,  where  significant:  sensors  controUed  by  sensor 
manager(s);  sensor  loading  and  power  management;  sensor,  processor  and  platform 
reliability;  sensor  processing  and  delays  due  to  sensors.  An  overall  score  of  "meets" 
guidelines  requires  that  the  model  should  be  able  to  depict  sensor  performance  with 
minor  simphfying  assumptions,  based  upon  environment  and  sensor  capabihty.  Field  of 
view,  field  of  regard,  and  slew  rates  are  modeled.  Sensors  are  controlled  by  sensor 
manager  when  necessary.  An  overall  score  of  "minor  problems  noted"  should  be  used 
sensor  functionahty  is  over-simphfied  funchon  of  environments  and  capabihhes.  Effective 
use  of  multiple  sensor  or  overloading  of  narrow  field  of  view  sensors  is  not  addressed. 
An  overaU  score  of  "major  problems  noted"  should  be  used  if  the  model  does  not 
include  major  sensor  types. 

e.  Tracking  and  Data  Fusion 

Threat  object  trajectory  (current,  projected,  and  in  some  cases,  original)  is  a  principal 
input  to  ahocation  and  engagement  functions;  it  must  be  done  to  the  level  of  fidehty 
required  within  the  simulation  to  provide  a  credible  basis  for  subsequent  determinations. 
An  overah  score  of  "exceeds"  guidelines  requires  that  the  simulation  model  should  be 
able  to  form  credible  tracks  for  threat  objects  based  on  sensor  capabilities  and  fusion  of 
such  data  from  ah  sensors  viewing  the  threat.  Tracks  should  include  error  covariances 
for  position  and  velocity.  Filter(s)  should  be  provided  to  refine  tracks  and  predict 
trajectory,  launch,  and  impact  points.  An  overah  score  of  "meets"  guidelines  requires 
that  the  model  be  able  to  form  tracks  based  upon  sensor  capabihty,  and  consider  data 
from  ah  sensors  to  reduce  uncertainties.  Tracks  need  to  include  position  co- variance 
error  and  impact  point  prediction.  An  overah  score  of  "minor  problems  noted"  should 
be  used  if  track  formation  is  solely  based  on  time  of  viewing  of  the  threat  and/or  track 
quahty  is  based  on  sensor  type  only  and  not  on  geometries.  Track  prediction  uncertainty 
is  not  provided.  An  overall  score  of  "major  problems  noted"  should  be  used  when  the 
tracks  are  based  on  perfect  knowledge. 
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f.  Surveillance  and  Intelligence 

Long  term  and  global  information  provides  the  background  for  situation  assessment- 
Modeting  should  at  desired  time  requires  information  management.  An  overall  score  of 
"exceeds  guidelines"  requires  that  aU  indicate  quahty  and  timeliness  of  information. 
Provision  of  correct  information  to  appropriate  game  elements  sources  of  information 
included  in  the  model  should  reflect  the  effects  of  overload,  bottlenecks,  environmental 
variations,  conflicting  information  content,  data  fusion  and  so  on.  A  capabihty  is 
provided  to  model  false  information  on  decoys  and  misidentifled  elements.  An  overall 
score  of  "meets  guidelines"  requires  that  the  impact  of  principal  information  sources  is 
exphcitiy  modeled.  Delays  are  accounted  for.  An  overall  score  of  "minor  problems 
noted"  indicates  that  some  information  necessary  for  decisions  is  not  exphcitiy  modeled, 
is  instantaneously  available  to  all  assets,  or  has  no  impact  on  the  decision  process.  An 
overall  score  of  "major  problems  noted"  indicates  that  surveiUance  and  intehigence 
information  is  not  exphcitiy  considered. 

g.  Resolution  and  Discrimination 

Determination  of  the  number  of  threat  objects  (resolution)  and  the  discrimination  of  real 
targets  from  associated  objects  (discrimination)  is  a  vital  part  of  the  ballistic  missile 
defense,  affecting  both  the  use  and  waste  of  interceptors  and  the  quality  of  sensors 
needed.  An  overah  score  of  "exceeds"  guidelines  requires  that  the  simulation  be  able  to 
resolve  the  number  of  objects  based  on  accuracy  of  sensor  sensitivities,  range  to 
objects,  background,  dispersion,  and  size  of  threat  objects.  Also  able  to  model 
discrimination  of  threat  objects  based  on  the  actual  capabihty  of  the  sensor.  Sensors 
controhed  so  that  they  wih  continue  to  view  the  object  until  resolution  and  discrimination 
is  complete.  Uses  physical  discriminators  (i.e.,  tumble  rate,  etc.).  An  overah  score  of 
"meets"  guidelines  requires  that  the  model  be  able  to  model  resolution  based  on  sensor 
sensitivities,  range,  and  dispersion  of  threat  objects.  Able  to  discriminate  objects  based 
on  sensor  time  of  viewing  and  parametric  description  of  threat  object  signatures.  An 
overah  score  of  "minor  problems  noted"  should  be  used  if  track  formation  is  only  able  to 
model  the  effects  of  resolution  and  discrimination.  An  overall  score  of  "major  problems 
noted"  should  be  used  when  the  model  does  not  consider  discrimination  or  resolution. 

2.  BIAS  (LACK  OF). 

a.  Vahdity  of  Assumptions 

How  far  removed  from  “real  world”  is  the  simulation  due  to  assumptions? 

b.  Sensitivity  to  Scenario 

How  sensitive  is  the  simulation  to  scenarios? 

c.  Use  of  Randomness 

How  adequately  is  randomness  handled? 

How  are  random  numbers  and  probabilities  used? 
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How  is  Monte  Carlo  handled? 

Are  there  separate  random  streams  by  function  or  only  one? 


3.  FLEXIBILITY/ADAPTABILITY 

a.  Ease  of  Modification 

How  adequately  is  modification  of  software  handled? 

How  easily  can  the  software  be  modified  for  different  analyses? 

Is  the  user  permitted  to  modify  the  source  code? 

b.  Ease  of  Growth  of  Capability 

How  adequately  is  growth  capabihty  handled? 

How  good  is  the  simulation’s  potential  for  growth? 

How  good  is  the  simulation’s  acceptance  of  new  algorithms? 

Is  the  user  permitted  to  add  or  change  features? 

c.  Elexibihty  in  Representation 

How  adequately  is  flexibihty  of  representation  handled? 

How  well  does  the  software  lend  itself  to  large  threat  analysis? 

How  well  does  the  software  lend  itself  to  long  duration  engagements? 
How  well  does  the  software  lend  itself  to  architecture  trades? 

d.  Elexibihty  in  Hardware  Requirements 

How  flexible  is  the  model  in  terms  of  hardware  requirements? 

How  many  different  computers  will  it  currently  operate  on? 

Is  the  model  portable  to  computers  not  currently  supported? 


58 


4. 1 1  APPENDIX  B  -  INTEGRA TION/DA TA  MANAGEMENT  EVALUA TION 
MATRICES 


Key  to  rating  symbols: 
n  =  not  applicable 
e  =  exceeds  guidelines 
m  =  meets  guidelines 
p  =  minor  problems  noted 
P  =  major  problems  noted 
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OPERATIONAL  EFFECTIVENESS  EVALUATION  MATRIX 
INTEGRATION/DATA  MANAGEMENT  _ 


MODEL:  NATE  (STRATC2AM/SGP4  &  SDP4)  _ 


INTENDED  USE:  Evaluation  of  methods  for  disseminating  missile  warning  message  traffic. 


1,  MODEL  COMPATIBILITY 


a.  Assumptions 


b.  Limitations 


c.  Physical  Parameters 


d.  Fidelity 


2,  INTEGRATION 


a.  Model/Framework  Interfaces 


b.  Model/Model  Interfaces 


c.  User  Interface/Input  &  Output 


d.  Runtime  Data  Management 


e.  Model  Modification  Interfaces 


3.  DATA  MANAGEMENT 


a.  Data  Preparation 


b.  Data  Collection 


c.  Data  Processing/Analysis 


d.  Data  Presentation 


COMMENTS:  Checked  Omni  results  against  examples 
found  in  Project  Spacetrack  report.  Overall  agreement 
between  results  is  not  bad,  but  there  were  differences 
between  the  two.  For  the  low  altitude  example,  the 
position  vectors  given  by  Omni  were  off  by  about  1.3km  in 
magnitude  from  the  values  that  the  Spacetrack  report  said 
they  should  be.  For  the  high  altitude  example,  the  position 
vectors  were  off  by  about  .02km.  Did  Autometrics  make  a 
bad  assumption  about  the  effects  of  using  lower  precision 
in  their  orbital  descriptions. 


COMMENTS:  Omni  uses  lower  precision  numbers  for 
EPOCH  &  MEAN  MOTION  orbital  parameters  than  the 
standard  two-card  element  set  format  calls  for  and  SGP4 
&  SDP4  were  designed  for.  Omni  uses  3  less  decimal 
places  for  EPOCH  and  1  less  decimal  place  for  MEAN 
MOTION.  There  doesn't  appear  to  be  any  reason  for  this 
other  than  a  mistake  in  the  design  of  Omni's  user 
interface. 


COMMENTS:  Data  is  handled  fine  once  it  gets  into  the 
system,  but  Omni's  user  interface  will  not  allow  the  user  to 
input  orbital  parameters  with  the  same  precision  that  the 
standard  two-card  element  set  format  dictates. 


Figure  18  -  Integration/Data  Management  Evaluation  Matrix  (Interface  1 ) 
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OPERATIONAL  EFFECTIVENESS  EVALUATION  MATRIX 
INTEGRATION/DATA  MANAGEMENT 


MODEL:NATE  (STRATC2AM  to  Omni) _ 


INTENDED  USE:Evaluation  of  methods  for  disseminating  missiie  warning  message  traffic. 


1.  MODEL  COMPATIBILITY 


a.  Assuptions 


b.  Limitations 


c.  Physical  Parameters 


d.  Fidelity 


2.  INTEGRATION 


a.  Model/Framework  Interfaces 


b.  Model/Model  Interfaces  _ 


c.  User  Interface/Input  &  Output 


d.  Runtime  Data  Management 


e.  Model  Modification  Interfaces 


3.  DATA  MANAGEMENT  _ 


a.  Data  Preparation 


b.  Data  Collection  _ 


c.  Data  Processing/Analysis 


d.  Data  Presentation 


COMMENTS:Omni  is  acting  as  a  graphical  user  interface 
to  STRATC2AM.  Therefore,  criteria  concerned  with 
compatibility  between  models  is  not  appropriate. 


COMMENTS:Comments  on  STRATC2AM  data  input  are 
not  appropriate  here.  There  were  no  problems  noted  with 
Omni's  data  management  and  user  interface.  Since 
STRATC2AM  is  distributed  with  its  source  code,  it  is 
certainly  open  to  user  modifications.  Whether  or  not  the 
existing  documentation  is  conducive  to  user  modifications 
is  another  story.  Omni  is  not  open  to  user  modifications, 
but  it  is  a  commercial  product  and  it  shouldn’t  be  expected 
to  be  open  to  modifications. 


COMMENTS:The  link  filtering  and  link  shading  features  in 
Omni  are  very  effective  tools  for  selectively  viewing 
STRATC2AM  results  in  animations.  However,  Omni's 
data  plotting  of  STRATC2AM  postprocessor  reports  leaves 
something  to  be  desired.  Continuous  line  graphs  can  be 
misleading  in  many  cases.  Also,  the  Grouped  Packet 
Receipt  report  did  not  appear  to  be  plotted  correctly  at  all. 


Figure  19  -  Integration/Data  Management  Evaluation  Matrix  (Interface  2) 
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OPERATIONAL  EFFECTIVENESS  EVALUATION  MATRIX 
INTEGRATION/DATA  MANAGEMENT 


MODELiNATE  (Omni  to  STRATC2AM) _ 


INTENDED  USEiEvaluation  of  methods  for  disseminating  missile  warning  message  traffic. 


COMMENTS:Omni  is  acting  as  a  graphical  user  interface 
to  STRATC2AM.  Therefore,  criteria  concerned  with 
compatibility  between  models  is  not  appropriate. 


COMM  ENTS  Tried  created  fixed,  moving,  satellite,  and 
missile  nodes  in  Omni.  Transferred  to  STRATC2AM  both 
by  creating  new  node  def  file  and  appending  to  existing 
file.  With  the  exception  of  changing  some  location  values 
a  negligible  amount,  fixed  and  satellite  nodes  were 
transferred  fine.  The  moving  node  was  interpreted  by 
STRATC2AM  as  a  launched  node  and  missile  nodes 
cannot  currently  be  transferred  to  STRATC2AM. 


COMMENTS;Preprocessor  input  through  Omni  is  limited. 
Omni  allows  the  creation  of  nodes.  All  other  data  entry 
must  be  done  in  STRATC2AM  preprocessor.  Also,  use  of 
"platforms"  is  a  somewhat  awkward  interface  between 
Omni  and  STRATC2AM. 


Figure  20  -  Integration/Data  Management  Evaluation  Matrix  (Interface  3 ) 
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OPERATIONAL  EFFECTIVENESS  EVALUATION  MATRIX 
INTEGRATION/DATA  MANAGEMENT _ 


MODEL:  NATE  (STRATC2AM  User  Interface) _ 


INTENDED  USE:  Evaluation  of  methods  for  disseminating  missile  warning  message  traffic. 


COMMENTS:  There  are  no  model  compatibility  issues  in 
this  case.  Vt/e're  looking  at  the  interface  between 
STRATC2AM  and  the  user. 


COMMENTS:  STRATC2AM's  preprocessor  user  interface 
is  adequate  functionally,  but  it  is  definitely  not 
user-friendly.  The  command  line  interface  makes  the  data 
input  process  a  very  long  process.  The  scenario  creation 
process  is  much  too  human  resource  intensive  to  be  a 
selling  point.  Also,  inconsistencies  in  the  user  interface 
can  be  very  frustrating.  Different  keys  must  be  pressed  for 
the  same  actions  depending  on  where  the  user  is  in  the 
interface. 


COMMENTS:  Same  comments  as  above. 


Figure  21  -  Integration/Data  Management  Evaluation  Matrix  (Interface  4) 
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4. 12  APPENDIX  C  -  SPACE  C3  PLANNING  EVALUATION  MATRIX 


Key  to  rating  symbols: 
n  =  not  applicable 
e  =  exceeds  guidelines 
m  =  meets  guidelines 
p  =  minor  problems  noted 
P  =  major  problems  noted 
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OPERATIONAL  EFFECTIVENESS  EVALUATION  MATRIX - 

SPACE  C3  PLANNING 


MODEL:  NATE 

INTENDED  USE:  EvdliiatiOri  uf  muIJiOdS  for  UiSSCiniridlinQ  rnis&ile  warning  Iraflic. 

[l.  FIDELITY 

COtiriMENTS:  Data  rate  not  acloctablc  lor  hardwira  links. 

■ 

P 

a.  co<n  muni  cation 

Time  sequence  (or  some  statistics  not  oorrecl.  No  real 

P 

b.  Elccloonic  wailare 

dalBcfiiin  rnamially.  Likawse  for  resolution  and 

Eg 

c.  Environ  men! 

discrimination.  Tracking,  data  ftision.  surveillance  and 

p 

d.  Dotbction 

intellinenc^  aren't  ir>  NATE  and  cannot  be  be  einulaled  by 
user. 

p 

e.  Tracking  aivt  Data  Fusion 

p 

f,  SurvBillanc«3  and  InlBlIigorica 

p 

g.  Rcsolulion  and  Discrimination 

2.  BIAS  {lack  i>f) 

COMMENTS:  Assumptions  that  antennas  a/a  aMiays 
pornung  In  the  right  direction  am  unmalistic.  NATE  is  quite 
sensilivE  {flexible)  tn  soenanos.  Probability  of  failure  of 
equ  ipme  nt  is  specified  for  enti  re  eq  ui  pm  e  nl  classes.  If  one 
in  any  class  I  a  Us.  all  in  mat  class  fails  simultaneously. 

1 

a.  Validity  of  Assam p(fn ns 

□ 

b.  Sensitivity  to  Scenario 

P 

Q.  Use  uf  RanduiTinOsS 

P 

d.  Use  of  Rcl.  of  Eqpmt 

3. 

FLEX  IB  ILITY/ADAPTABI LITY 

COMMENTS;  Omni  is  not  easily  modified,  because  It  Is 
commercial  software.  Howevc/.  STRATC2AM  is 
distributed  witti  the  soutrx/  onde  and  its  growth  capability  Is 
ilJuslrated  by  its  20  year  history  of  growth.  The  flexibility  of 
NATE'S  design  makes  it  a  viable  tool  tor  C3  arcliitadure 
trade-offs,  but  the  amount  ol  detail  in  the  mudat  m ekes  it 
difficult  to  use  tor  long  duration  sccnartus.  NATE  is 
restricted  to  silicon  Graphics  computorr^  bec/r.ise  of  Omni, 
bul  STRATC2AM  Car  also  bp  ran  on  Suns.  In  the  Sun's 
nasE,  AXIS  would  be  substiiuted  for  Omni.. 

p| 

a.  Ease  ot  Uodiflcatiun 

□ 

0.  Ease  of  Growtn  or  Capability 

- 

c.  Flexibility  in  Representation 

P 

d.  Flexibility  in  Haiitware  ReguirBmerits 

M 

Figure  22  -  Space  C3  Planning  Evaluation  Matrix 
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5.0  SUMMARY 

This  report  described  the  results  of  three  different  initiatives  pursued  under  the  umbrella 
of  one  AFRL/IF  in-house  effort. 

When  this  effort  began,  its  objective  was  to  develop  strategies  for  routing  messages  in 
the  next  generation  MILSATCOM  environment.  Towards  this  objective,  several  routing 
algorithms  were  produced  in  a  rapid  prototyping  fashion  using  a  mle-based  approach.  The  first 
generation  of  algorithms  developed  was  tested  successfully  in  a  static  network  environment. 
The  second  generation  of  algorithms  was  never  tested  due  to  a  redirection  of  this  overall  effort. 
Testing  of  the  algorithms  in  a  dynamic  environment  was  not  done  for  the  same  reason. 

The  second  thrust  of  this  effort  was  to  perform  several  ATM  related  tasks  in  support  of 
theatre  extension  objectives  of  the  Global  Grid  concept.  These  tasks  were  interrupted  for 
approximately  a  year  to  pursue  the  NATE  vahdation  initiative  under  this  effort.  As  a  result, 
some  of  the  ATM  related  tasks  were  overtaken  by  events  and  were  canceled  because  they 
were  no  longer  relevant.  This  report  described  the  results  of  all  those  tasks.  In  the  cases  where 
the  tasks  ended  up  being  canceled,  partial  results  are  presented. 

The  third  thmst  of  this  effort  was  to  perform  a  prehminary  vahdations  and  accreditation 
of  USSPACECOM’s  NATE  simulation  model.  The  results  of  this  initiative  showed 
USSPACECOM  that  their  NATE  model  was  basically  a  very  sound  communications  simulation 
model.  Almost  all  of  the  mistakes  that  were  found  in  the  design  were  relatively  minor  and 
fixable.  USSPACECOM  has  since  had  these  mistakes  corrected. 
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